- Preparing search index...
- The search index is not available
If true, connect to multiple S3 files simultaneously to pre-fetch files when reading from a queue that has events stored in S3. This is a new feature and so is not on by default, though it can dramatically improve read performance. It is expected to be made the default in Q3 2022.
When using the ReadOptions.fast_s3_read feature, this specifies how many bytes of s3 data we want
to prefetch. The default usually is correct.
When using the ReadOptions.fast_s3_read feature, this specifies how many retries for S3 timeouts
before giving up on preconnecting.
The default usually is correct.
The limit of the number of records, events, to read in total from the queue before closing the read stream.
The max number of times the SDK will query the stream for new events before it shuts down the read stream. Consider that each query by the SDK to retrieve events from DynamoDB could return many, many events and that an "event" may actually be a pointer to an S3 file full of events.
It is uncommon for developers to have to set this, much less know it exists.
The largest event ID that you should read, exclusive. So, stop reading when you arrive at this event ID.
Use startTime and this option to read a range of events from a queue.
The duration of time the to read for before closing the read stream. It is common to set this to 75% to 80% of the time remaining before the lambda is shut down to give the lambda sufficient time to finish processing. Of course, different types of processing will differ.
Note, this type is any one of the valid durations the Moment JS library can take: Duration | number | string | FromTo | DurationInputObject.
The read stream will shutdown as soon as one of the constraints is met: runTime, loops, limit, size,
stopTime.
The limit of the number of bytes to read in total from the queue before closing the read stream.
The event ID of the starting position to read from with in the queue. It is common to not provide this because each queue is stateful in that it remembers the last read position of a bot. Then, as bots read they make a call back to the RStreams Bus to update the read position.
Usually, the SDK just handles this for you. So, if the start isn't provided, the SDK will just use the bot's last read position as the starting point. So, as bots are invoked, read some events and do some processing, they automatically update back how far they've read to and then the bot shuts down after a period of time. When the bot starts back up to read again, it knows where it last read from and just continues.
When to stop reading as a time since the epoch.
The max number of records, events, the SDK should retrieve each time it retrieves events from the RStreams Bus' Dynamo DB events table.
Generated using TypeDoc
Options when reading data from an instance of the RStreams bus. The options in this interface provide a lot of control and performance optimization options and developers should familiarize themselves with them. They are used in a read pipeline step to configure how to read.
Most bots are based on AWS lambda and lambdas can only run continuosly for 15 minutes. So, a bot that sets itself up to read events from a queue has to end at some time. Depending on how the bot registered itself with the bus, after a bot shuts down the bot will either be re-invoked by the RStreams bus when there are new events to be read or on whatever timer established.