| Type: | Package |
| Title: | 'AWS Python SDK' ('boto3') for R |
| Description: | Fork-safe, raw access to the 'Amazon Web Services' ('AWS') 'SDK' via the 'boto3' 'Python' module, and convenient helper functions to query the 'Simple Storage Service' ('S3') and 'Key Management Service' ('KMS'), partial support for 'IAM', the 'Systems Manager Parameter Store' and 'Secrets Manager'. |
| SystemRequirements: | Python and boto3 (https://aws.amazon.com/sdk-for-python) |
| Version: | 0.4.1 |
| Date: | 2025-01-23 |
| URL: | https://daroczig.github.io/botor/ |
| BugReports: | https://github.com/daroczig/botor/issues |
| RoxygenNote: | 7.3.2 |
| License: | AGPL-3 |
| Encoding: | UTF-8 |
| Imports: | utils, reticulate, checkmate, logger, jsonlite |
| Suggests: | testthat, covr, digest |
| NeedsCompilation: | no |
| Packaged: | 2025-01-24 12:50:04 UTC; daroczig |
| Author: | Gergely Daróczi 
[aut, cre],
System1 [fnd] |
| Maintainer: | Gergely Daróczi <daroczig@rapporter.net> |
| Repository: | CRAN |
| Date/Publication: | 2025-01-24 13:20:02 UTC |
Base64-decode a string into raw bytes using Python's base64 module
Description
Base64-decode a string into raw bytes using Python's base64 module
Usage
base64_dec(text)
Arguments
text |
string |
Value
raw bytes
See Also
Examples
## Not run:
botor:::base64_dec(botor:::base64_enc(charToRaw('foobar')))
## End(Not run)
Base64-encode raw bytes using Python's base64 module
Description
Base64-encode raw bytes using Python's base64 module
Usage
base64_enc(text)
Arguments
text |
|
Value
string
See Also
Raw access to the boto3 module imported at package load time
Description
Raw access to the boto3 module imported at package load time
Usage
boto3
Format
An object of class python.builtin.module (inherits from python.builtin.object) of length 0.
Note
You may rather want to use botor instead, that provides a fork-safe boto3 session.
boto3 version
Description
boto3 version
Usage
boto3_version()
Value
string
The default, fork-safe Boto3 session
Description
The default, fork-safe Boto3 session
Usage
botor(
aws_access_key_id,
aws_secret_access_key,
aws_session_token,
region_name,
botocore_session,
profile_name
)
Arguments
aws_access_key_id |
AWS access key ID |
aws_secret_access_key |
AWS secret access key |
aws_session_token |
AWS temporary session token |
region_name |
Default region when creating new connections |
botocore_session |
Use this Botocore session instead of creating a new default one |
profile_name |
The name of a profile to use. If not given, then the default profile is used |
Value
boto3 Session
Creates an initial or reinitialize an already existing AWS client or resource cached in the package's namespace
Description
Creates an initial or reinitialize an already existing AWS client or resource cached in the package's namespace
Usage
botor_client(service, type = c("client", "resource"), cache = TRUE, ...)
Arguments
service |
string, eg S3 or IAM |
type |
AWS service client or resource to be created, eg
|
cache |
booelan flag for caching the client or resource in the
package namespace. For (internal) package functions, it's best
to set to |
... |
further parameters passed to the |
Value
cached AWS client
References
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html
Internal boto3 session
Description
Internal boto3 session
Usage
botor_session
Format
An object of class NULL of length 0.
Look up the PID used to initialize the Boto3 session
Description
Look up the PID used to initialize the Boto3 session
Usage
botor_session_pid()
Value
int
Look up the UUID of the initialized Boto3 session
Description
Look up the UUID of the initialized Boto3 session
Usage
botor_session_uuid()
Value
int
Check if an argument looks like an S3 bucket
Description
Check if an argument looks like an S3 bucket
Usage
check_s3_uri(x)
Arguments
x |
string, URI of an S3 object, should start with |
Examples
check_s3_uri('s3://foo/bar')
check_s3_uri('https://foo/bar')
## Not run:
assert_s3_uri('https://foo/bar')
## End(Not run)
boto3 clients cache
Description
boto3 clients cache
Usage
clients
Format
An object of class environment of length 0.
Transforms a python2 string literal or python3 bytes literal into an R string
Description
This is useful to call eg for the KMS call, where python2 returns a string, but python3 returns bytes literals – calling "decode" is tricky, but bytearray conversion, then passing the raw vector to R and converting that a string works.
Usage
coerce_bytes_literals_to_string(x)
Arguments
x |
string |
Value
string
The default, fork-safe IAM client on the top of botor
Description
The default, fork-safe IAM client on the top of botor
Usage
iam()
Value
botocore.client.IAM
References
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/iam.html
Retrieves information about the specified IAM user, including the user's creation date, path, unique ID, and ARN
Description
Retrieves information about the specified IAM user, including the user's creation date, path, unique ID, and ARN
Usage
iam_get_user(...)
Arguments
... |
optional extra arguments passed |
Value
list
References
Get the current AWS username
Description
Get the current AWS username
Usage
iam_whoami()
Value
string
See Also
The default, fork-safe Kinesis client on the top of botor
Description
The default, fork-safe Kinesis client on the top of botor
Usage
kinesis()
Value
botocore.client.Kinesis
References
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/kinesis.html
Describes the specified Kinesis data stream
Description
Describes the specified Kinesis data stream
Usage
kinesis_describe_stream(stream)
Arguments
stream |
the name of the stream to describe |
Value
list
References
Gets data records from a Kinesis data stream's shard
Description
Gets data records from a Kinesis data stream's shard
Usage
kinesis_get_records(shard_iterator, limit = 25L)
Arguments
shard_iterator |
the position in the shard from which you want
to start sequentially reading data records, usually provided by
|
limit |
maximum number of records to return |
Value
list of Records, NextShardIterator and
MillisBehindLatest
References
Examples
## Not run:
botor(profile_name = 'botor-tester')
iterator <- kinesis_get_shard_iterator(stream = 'botor-tester', shard = '0')
kinesis_get_records(iterator$ShardIterator)
## End(Not run)
Gets an Amazon Kinesis shard iterator
Description
Gets an Amazon Kinesis shard iterator
Usage
kinesis_get_shard_iterator(
stream,
shard,
shard_iterator_type = c("TRIM_HORIZON", "LATEST", "AT_SEQUENCE_NUMBER",
"AFTER_SEQUENCE_NUMBER", "AT_TIMESTAMP"),
...
)
Arguments
stream |
the name of the stream to describe |
shard |
the shard ID of the Kinesis Data Streams shard to get the iterator for |
shard_iterator_type |
determines how the shard iterator is used to start reading data records from the shard |
... |
optional further parameters, such as
|
Value
list of ShardIterator
References
See Also
Writes a single data record into an Amazon Kinesis data stream
Description
Writes a single data record into an Amazon Kinesis data stream
Usage
kinesis_put_record(stream, data, partition_key, ...)
Arguments
stream |
the name of the stream to describe |
data |
the data blob (<1 MB) to put into the record, which is base64-encoded when the blob is serialized |
partition_key |
Unicode string with a maximum length limit of 256 characters determining which shard in the stream the data record is assigned to |
... |
optional further parameters, such as
|
Value
list of ShardId, SequenceNumber and
EncryptionType
References
The default, fork-safe KMS client on the top of botor
Description
The default, fork-safe KMS client on the top of botor
Usage
kms()
Value
botocore.client.KMS
References
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/kms.html
Decrypt cipher into plain text via KMS
Description
Decrypt cipher into plain text via KMS
Usage
kms_decrypt(cipher, simplify = TRUE)
Arguments
cipher |
Base64-encoded ciphertext |
simplify |
returns decrypted plain-text instead of raw list |
Value
decrypted text as string or list
See Also
Decrypt file via KMS
Description
Decrypt file via KMS
Usage
kms_decrypt_file(file, return = file)
Arguments
file |
base file path (without the |
return |
where to place the encrypted file (defaults to |
Value
decrypted file path
See Also
Encrypt plain text via KMS
Description
Encrypt plain text via KMS
Usage
kms_encrypt(key, text, simplify = TRUE)
Arguments
key |
the KMS customer master key identifier as a fully
specified Amazon Resource Name (eg
|
text |
max 4096 bytes long string, eg an RSA key, a database password, or other sensitive customer information |
simplify |
returns Base64-encoded text instead of raw list |
Value
string or list
See Also
Encrypt file via KMS
Description
Encrypt file via KMS
Usage
kms_encrypt_file(key, file)
Arguments
key |
the KMS customer master key identifier as a fully
specified Amazon Resource Name (eg
|
file |
file path |
Value
two files created with enc (encrypted data) and
key (encrypted key) extensions
See Also
Generate a data encryption key for envelope encryption via KMS
Description
Generate a data encryption key for envelope encryption via KMS
Usage
kms_generate_data_key(key, bytes = 64L)
Arguments
key |
the KMS customer master key identifier as a fully
specified Amazon Resource Name (eg
|
bytes |
the required length of the data encryption key in
bytes (so provide eg |
Value
list of the Base64-encoded encrypted version of the
data encryption key (to be stored on disk), the raw
object of the encryption key and the KMS customer master key
used to generate this object
Guess the type of a file based on the filename using mimetypes Python module
Description
Guess the type of a file based on the filename using mimetypes Python module
Usage
mime_guess(file)
Arguments
file |
path |
Value
string
Looks up major version of Python (eg 2 or 3)
Description
Looks up major version of Python (eg 2 or 3)
Usage
python_version()
Value
number
Imports and caches a Python module
Description
Imports and caches a Python module
Usage
require_python_builtins()
Value
imported Python module
Imports and caches a Python module
Description
Imports and caches a Python module
Usage
require_python_module(module)
Arguments
module |
a Python module name |
Value
imported Python module
The default, fork-safe Amazon Simple Storage Service (S3) client on the top of botor
Description
The default, fork-safe Amazon Simple Storage Service (S3) client on the top of botor
Usage
s3(disable_signing = getOption("botor-s3-disable-signing"))
Arguments
disable_signing |
boolean if requests should be signed. Set to |
Value
s3.ServiceResource
References
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#service-resource
Copy an object from one S3 location to another
Description
Copy an object from one S3 location to another
Usage
s3_copy(uri_source, uri_target)
Arguments
uri_source |
string, location of the source file |
uri_target |
string, location of the target file |
Value
invisibly uri_target
References
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Object.copy
Delete an object stored in S3
Description
Delete an object stored in S3
Usage
s3_delete(uri)
Arguments
uri |
string, URI of an S3 object, should start with |
Download a file from S3
Description
Download a file from S3
Usage
s3_download_file(uri, file, force = TRUE)
Arguments
uri |
string, URI of an S3 object, should start with |
file |
string, location of local file |
force |
boolean, overwrite local file if exists |
Value
invisibly file
References
Examples
## Not run:
s3_download_file('s3://botor/example-data/mtcars.csv', tempfile())
## End(Not run)
Checks if an object exists in S3
Description
Checks if an object exists in S3
Usage
s3_exists(uri)
Arguments
uri |
string, URI of an S3 object, should start with |
Value
boolean
Examples
## Not run:
s3_exists('s3://botor/example-data/mtcars.csv')
s3_exists('s3://botor/example-data/UNDEFINED.CSVLX')
## End(Not run)
List all S3 buckets
Description
List all S3 buckets
Usage
s3_list_buckets(simplify = TRUE)
Arguments
simplify |
return bucket names as a character vector |
Value
list of boto3.resources.factory.s3.Bucket or a character vector
List objects at an S3 path
Description
List objects at an S3 path
Usage
s3_ls(uri)
Arguments
uri |
string, should start with |
Value
data.frame with bucket_name, object
key, uri (that can be directly passed to eg
s3_read), size in bytes, owner and
last_modified timestamp
Create an S3 Object reference from an URI
Description
Create an S3 Object reference from an URI
Usage
s3_object(uri)
Arguments
uri |
string, URI of an S3 object, should start with |
Value
s3$Object
Sets tags on s3 object overwriting all existing tags. Note: tags and metadata tags are not the same
Description
Sets tags on s3 object overwriting all existing tags. Note: tags and metadata tags are not the same
Usage
s3_put_object_tagging(uri, tags)
Arguments
uri |
string, URI of an S3 object, should start with
|
tags |
named character vector, e.g. |
References
Download and read a file from S3, then clean up
Description
Download and read a file from S3, then clean up
Usage
s3_read(uri, fun, ..., extract = c("none", "gzip", "bzip2", "xz"))
Arguments
uri |
string, URI of an S3 object, should start with |
fun |
R function to read the file, eg |
... |
optional params passed to |
extract |
optionally extract/decompress the file after downloading from S3 but before passing to |
Value
R object
Examples
## Not run:
s3_read('s3://botor/example-data/mtcars.csv', read.csv)
s3_read('s3://botor/example-data/mtcars.csv', data.table::fread)
s3_read('s3://botor/example-data/mtcars.csv2', read.csv2)
s3_read('s3://botor/example-data/mtcars.RDS', readRDS)
s3_read('s3://botor/example-data/mtcars.json', jsonlite::fromJSON)
s3_read('s3://botor/example-data/mtcars.jsonl', jsonlite::stream_in)
## read compressed data
s3_read('s3://botor/example-data/mtcars.csv.gz', read.csv, extract = 'gzip')
s3_read('s3://botor/example-data/mtcars.csv.gz', data.table::fread, extract = 'gzip')
s3_read('s3://botor/example-data/mtcars.csv.bz2', read.csv, extract = 'bzip2')
s3_read('s3://botor/example-data/mtcars.csv.xz', read.csv, extract = 'xz')
## End(Not run)
Split the bucket name and object key from the S3 URI
Description
Split the bucket name and object key from the S3 URI
Usage
s3_split_uri(uri)
Arguments
uri |
string, URI of an S3 object, should start with |
Value
list
Upload a file to S3
Description
Upload a file to S3
Usage
s3_upload_file(file, uri, content_type = mime_guess(file))
Arguments
file |
string, location of local file |
uri |
string, URI of an S3 object, should start with |
content_type |
content type of a file that is auto-guess if omitted |
Value
invisibly uri
References
See Also
Examples
## Not run:
t <- tempfile()
write.csv(mtcars, t, row.names = FALSE)
s3_upload_file(t, 's3://botor/example-data/mtcars.csv')
unlink(t)
## note that s3_write would have been a much nicer solution for the above
## End(Not run)
Write an R object into S3
Description
Write an R object into S3
Usage
s3_write(x, fun, uri, compress = c("none", "gzip", "bzip2", "xz"), ...)
Arguments
x |
R object |
fun |
R function with |
uri |
string, URI of an S3 object, should start with |
compress |
optionally compress the file before uploading to
S3. If compression is used, it's better to include the related
file extension in |
... |
optional further arguments passed to |
Note
The temp file used for this operation is automatically removed.
Examples
## Not run:
s3_write(mtcars, write.csv, 's3://botor/example-data/mtcars.csv', row.names = FALSE)
s3_write(mtcars, write.csv2, 's3://botor/example-data/mtcars.csv2', row.names = FALSE)
s3_write(mtcars, jsonlite::write_json, 's3://botor/example-data/mtcars.json', row.names = FALSE)
s3_write(mtcars, jsonlite::stream_out, 's3://botor/example-data/mtcars.jsonl', row.names = FALSE)
s3_write(mtcars, saveRDS, 's3://botor/example-data/mtcars.RDS')
## compress file after writing to disk but before uploading to S3
s3_write(mtcars, write.csv, 's3://botor/example-data/mtcars.csv.gz',
compress = 'gzip', row.names = FALSE)
s3_write(mtcars, write.csv, 's3://botor/example-data/mtcars.csv.bz2',
compress = 'bzip2', row.names = FALSE)
s3_write(mtcars, write.csv, 's3://botor/example-data/mtcars.csv.xz',
compress = 'xz', row.names = FALSE)
## End(Not run)
The default, fork-safe AWS Systems Manager (SecretManager) client on the top of botor
Description
The default, fork-safe AWS Systems Manager (SecretManager) client on the top of botor
Usage
sm()
Value
botocore.client.secretsmanager
References
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/secretsmanager.html
Read AWS System Manager's Secrets Manager via Secret Manager
Description
Read AWS System Manager's Secrets Manager via Secret Manager
Usage
sm_get_secret(path, key = NULL, parse_json = TRUE)
Arguments
path |
name/path of the key to be read |
key |
single key or a vector of keys. |
parse_json |
logical. Default TRUE |
Value
(optionally decrypted) value
The default, fork-safe AWS Systems Manager (SSM) client on the top of botor
Description
The default, fork-safe AWS Systems Manager (SSM) client on the top of botor
Usage
ssm()
Value
botocore.client.SSM
References
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ssm.html
Read AWS System Manager's Parameter Store
Description
Read AWS System Manager's Parameter Store
Usage
ssm_get_parameter(path, decrypt = TRUE)
Arguments
path |
name/path of the key to be read |
decrypt |
decrypt the value or return the raw ciphertext |
Value
(optionally decrypted) value
Returns details about the IAM user or role whose credentials are used to call the operation
Description
Returns details about the IAM user or role whose credentials are used to call the operation
Usage
sts_whoami()
Value
list with UserId, Account and Arn
References
See Also
Extract error message from a Python exception
Description
Extract error message from a Python exception
Usage
trypy(expression)
Arguments
expression |
R expression |
Value
error
Examples
## Not run:
trypy(botor()$resource('foobar'))
trypy(sum(1:2))
trypy(sum(1:1foo))
## End(Not run)
Generate UUID using Python's uuid module
Description
Generate UUID using Python's uuid module
Usage
uuid()
Value
string