- ×
60fps with Javascript Coroutines for idle processing and animation
Filed under dom › frameworksShow Alljs-coroutines
Supports all browsers and React Native
When is the right time to sort a massive array on the main thread of a Javascript app? Well any time you like if you don't mind the user seeing all of your animations and effects jank to hell. Even transferring to a worker thread is going to hit the main thread for serialization and stutter everything.
So when is the right time? Well it's in all those gaps where you animation isn't doing anything and the system is idle. If only you could write something to use up that time and then relinquish control to the system so it can animate and do the rest of the work, then resume in the next gap. Well now you can...
Get 60fps while sorting an array of 10 million items with js-coroutines
Quick Start
The project's main web site contains examples of js-coroutines in operation, explains how it can provide benefits to your project and has links to the full API docs plus some examples.
JS-COROUTINES Overview and API docs
How it works?
This dev.to article goes into detail about how js-coroutines works
Demo
See the Code Sandbox Demo.
Animating Using Coroutines
Another super useful way of using coroutines is to animate and control complex states - js-coroutines provides this too with the powerful
update
method that runs every frame in high priority.There's an example of how to write your own animation later and you can see this CodeSandbox demo of stateful animations, or this game built using js-coroutines, for more.
Commonly required asynchronous operations
You can use
*stringify()
and*parse()
to manipulate JSON in an idle coroutine that won't block the main thread.You can use
stringifyAsync()
andparseAsync()
to perform JSON parsing and stringifying anywhere you can take a promise orawait
a response.You can use
*compress()
and*decompress()
to compress to storable/transmittable strings.You can use
compressAsync()
anddecompressAsync()
to perform compression and decompression anywhere you can take a promise orawait
a response.Compression
js-coroutines uses lz-string for compression.
LZ-String GitHub/Documentation.
Installation
npm install --save js-coroutines
Usage
You can make your own generator functions that do anything you like and
yield
to check if there is time remaining this frame:import {run, sort, stringify} from 'js-coroutines' ... let json = await run(function*() { const results = []; for(let i = 0; i < 10000000; i++) { results.push(Math.random() * 10000); //Check how much time left every 100 entries if(i % 100 === 0) yield; } //Pass to a coroutine sort function yield* sort(results, value=>value) return yield* stringify(results); })
Or you can just use the Async helper functions in an async routine. This is less powerful, but you don't have to start writing generator functions or working out where to yield.
import { parseAsync, mapAsync } from "js-coroutines"; async function process(url) { const response = await fetch("someurl"); //Use the coroutine version of parse, rather than blocking //the main thread permanently by using .json() const result = await parseAsync(await response.text()); //Imagining the result is some database rows, map out the //desired response without blocking the main thread for paints const values = await mapAsync(result, (row) => ({ item: row.time, value: row.quantity * row.unitPrice, })); return values; }
Getting Started With Async Functions
Async functions are the easiest way to use js-coroutines if you just need to handle common functions like sorts, finds, filters and JSON parsing in the background. If you need to break up your own logic you will have to write a generator.
Just import the
xxxAsync
version of the function from js-coroutines and use a standard Promise chain orawait
and the code will run only in the gaps.async function asyncFunctions() { // Parse the JSON async let o = await parseAsync(json); // Concatenate arrays in the background for (let i = 1; i < 12; i++) { o = await concatAsync(o, o); } // Write out the arrays let output = await stringifyAsync(o); // Map ids from the array in the background let justIds = await mapAsync(o, (v) => v.id); // Return the JSON of just the ids return [output, await stringifyAsync(justIds)]; }
Getting Started With Function Pipelines
You can also create pure functional pipelines using pipe, tap, branch, repeat and call
import {pipe, parseAsync, tap, mapAsync} from 'js-coroutines' const process = pipe( parseAsync, function * (data) { let i = 0 let output = [] for(let item of data) { output.push({...item, total: item.units * item.price}) if((i++ % 100)==0) yield } return output }, mapAsync.with(v=>({value: v.total, item: v.item})), tap(console.log), stringifyAsync ) ... console.log(await process(data))
Getting Started Writing Your Own Generators
js-coroutines
uses generator functions andrequestIdleCallback
to let you easily split up your work with minimal effort.A simple generator:
await run(function* () { const strings = []; let results; //Create 2 million rows of random values results = new Array(2000000); for (let i = 0; i < 2000000; i++) { //Every 128th record, check to see if we still have time //run the remainder on another tick if we don't if ((i & 127) === 0) yield; results[i] = (Math.random() * 10000) | 0; } //Double all the values yield* forEach( results, yielding((r, i) => (results[i] = r * 2)) ); //Get the square roots const sqrRoot = yield* map( results, yielding((r) => Math.sqrt(r)) ); //Sum all of the items const sum = yield* reduce( results, yielding((c, a) => c + a, 64), 0 ); //Join the arrays yield* append(results, sqrRoot); // Sort the results yield* sort(results, (a, b) => a - b); return results; });
As you can probably see, it comes ready with the most useful functions for arrays:
forEach
map
filter
reduce
findIndex
find
some
every
sort
append
(array into array)concat
(two arrays into a new array)
The helper
yielding
wraps a normal function as a generator and checks remaining time every few iterations. You can see it in use above. It's just a helper though - if yourmap
function needs to do more work it can just be a generator itself, yield when it likes and also pass on to deeper functions that can yield:const results = yield * map(inputArray, function* (element, index) { //Every 200 indices give up work //on this frame by yielding 'true' //yield without true, checks the amount //of remaining time if (index % 200 === 199) yield true; //Yield out a filter operation let matched = yield* filter( element, yielding((c) => c > 1000) ); //Now yield out the calculation of a sum return yield* reduce( matched, yielding((c, a) => c + a), 0 ); });
yielding(fn, [optional yieldFrequency]) -> function *
##
Async
Former
runAsync
is deprecated. You may yield a Promise instead. js-coroutines will automatically restart the coroutine when the Promise is resolved.const results = await run( function* () { const response = yield fetch("http://someurl"); const rows = yield response.json(); yield* sort(rows, (a) => a.value); return processed; });
Update coroutines
A great way to do stateful animation is using a coroutine running every frame. In this case when you
yield
you get called back on the next frame making stateful animations a piece of cake:import { update } from "js-coroutines"; //Animate using a coroutine for state update(function* () { while (true) { //Move left to right for (let x = -200; x < 200; x++) { logoRef.current.style.marginLeft = `${x * multiplier}px`; yield; //Now we are on the next frame } //Move top to bottom for (let y = 0; y < 200; y++) { logoRef.current.style.marginTop = `${y * multiplier}px`; yield; } //Move diagonally back for (let x = 200; x > -200; x--) { logoRef.current.style.marginLeft = `${x * multiplier}px`; logoRef.current.style.marginTop = ((x + 200) * multiplier) / 2 + "px"; yield; } } });
Writing Coroutines with the API
run(coroutineFunction, msToLeaveSpare=1, timeout=160) -> TerminatablePromise(Any)
coroutineFunction
must be afunction *
Run your coroutine, which will occupy up to the last amount of m/s specified in the
msToLeaveSpare
(0.5 is the minimum) of the idle time on the thread.timeout
specifies the time before it will run if there is no idle time (default 1/10 frames).The promise returned has a
terminate(result)
function that can be used to stop the calculation early - maybe you want to go again with different parameters.yield
inside your coroutine will check how much time is left and continue if there is enough.yield 2
yielding a number results in a check for at least that number of ms remaining.yield true
will definitely abandon the current frames work. Useful if you are about to/just have allocated tons of memory to give time for GC.yield* generatorFn([param], [...param])
call a generator function which will take over yielding time checks and return the value it creates when done.function* myCoroutine() { const results = []; for (let i = 1; i < 1000000; i++) { if ((i & 127) === 0) yield; //time check results.push(i); } yield true; // end current frame processing let anotherArray = new Array(results.length); yield true; // give time for GC // Run a for loop on the results yield* forEach( results, yielding( (result, index, collection) => (anotherArray[index] = result / collection.length) ) ); return anotherArray; }
*yielding(fn, [optional frequency=8]) -> function *
Converts a normal function into one that yields every
frequency
calls.Very useful for providing map/filter functions etc.
wrapAsPromise(coroutine) -> function([params]) -> Promise(Any)
Returns an async function that can be called with await and will call the passed in coroutine forwarding parameters.
//Create an async function const toTuplesAsync = wrapAsPromise(function* (array) { let output = []; //Create tuples for (let i = 0; i < array.length; i += 2) { output.push([array[i], array[i + 1]]); yield; } return output; }); ... async function myProcess() { const data = await getDataFromSomewhere(); //Call your wrapped coroutine const tuples = await toTuplesAsync(data); //do something with the result return processTuplesSomehow(tuples); }
License
js-coroutines - MIT (c) 2020 Mike Talbot
Timsort - MIT (c) 2015 Marco Ziccardi (c) 2020 Mike Talbot (Generator modifications)
JSON stringify - Public Domain (c) 2017 Douglas Crockford (c) 2020 Mike Talbot (Generator modifications)
JSON Parse - yastjson - MIT (c) 2020 5u9ar (zhuyingda) (c) 2020 Mike Talbot (Optimisations and generator modifications)