s3

class moto.s3.models.S3Backend(region_name: str, account_id: str)

Custom S3 endpoints are supported, if you are using a S3-compatible storage solution like Ceph. Example usage:

os.environ["MOTO_S3_CUSTOM_ENDPOINTS"] = "http://custom.internal.endpoint,http://custom.other.endpoint"

@mock_aws
def test_my_custom_endpoint():
    boto3.client("s3", endpoint_url="http://custom.internal.endpoint")
    ...

Note that this only works if the environment variable is set before the mock is initialized.

_-_-_-_

When using the MultiPart-API manually, the minimum part size is 5MB, just as with AWS. Use the following environment variable to lower this:

S3_UPLOAD_PART_MIN_SIZE=256

_-_-_-_

When listing objects, the default max-keys value is 1000, just as with AWS. Use the following environment variable to configure this this:

MOTO_S3_DEFAULT_MAX_KEYS=256

_-_-_-_

CrossAccount access is allowed by default. If you want Moto to throw an AccessDenied-error when accessing a bucket in another account, use this environment variable:

MOTO_S3_ALLOW_CROSSACCOUNT_ACCESS=false

_-_-_-_

Install moto[s3crc32c] if you use the CRC32C algorithm, and absolutely need the correct value. Alternatively, you can install the crc32c dependency manually.

If this dependency is not installed, Moto will fall-back to the CRC32-computation when computing checksums.

_-_-_-_

S3 has two endpoint styles, where the bucket name can be either part of the domain (bucket.s3.amazonaws.com) or part of the path (s3.amazonaws.com/bucket).

If you’re running MotoServer on a custom host, for instance on http://moto.service:5000, a CreateBucket-request then becomes http://moto.service:5000/bucketname.

However, because Moto tries to get the bucket-name from the domain first, it will think that ‘moto’ is the bucket name.

You can disable this behaviour with an environment variable, so that Moto will always look at the path instead:

S3_IGNORE_SUBDOMAIN_BUCKETNAME=true

Implemented features for this service

  • [X] abort_multipart_upload

  • [X] complete_multipart_upload

  • [X] copy_object

  • [X] create_bucket

  • [ ] create_bucket_metadata_configuration

  • [ ] create_bucket_metadata_table_configuration

  • [X] create_multipart_upload

  • [ ] create_session

  • [X] delete_bucket

  • [ ] delete_bucket_analytics_configuration

  • [X] delete_bucket_cors

  • [X] delete_bucket_encryption

  • [ ] delete_bucket_intelligent_tiering_configuration

  • [ ] delete_bucket_inventory_configuration

  • [X] delete_bucket_lifecycle

  • [ ] delete_bucket_metadata_configuration

  • [ ] delete_bucket_metadata_table_configuration

  • [ ] delete_bucket_metrics_configuration

  • [X] delete_bucket_ownership_controls

  • [X] delete_bucket_policy

  • [X] delete_bucket_replication

  • [X] delete_bucket_tagging

  • [X] delete_bucket_website

  • [X] delete_object

  • [X] delete_object_tagging

  • [X] delete_objects

  • [X] delete_public_access_block

  • [ ] get_bucket_abac

  • [X] get_bucket_accelerate_configuration

  • [X] get_bucket_acl

  • [ ] get_bucket_analytics_configuration

  • [X] get_bucket_cors

  • [X] get_bucket_encryption

  • [ ] get_bucket_intelligent_tiering_configuration

  • [X] get_bucket_inventory_configuration

  • [X] get_bucket_lifecycle

  • [X] get_bucket_lifecycle_configuration

  • [X] get_bucket_location

  • [X] get_bucket_logging

  • [ ] get_bucket_metadata_configuration

  • [ ] get_bucket_metadata_table_configuration

  • [ ] get_bucket_metrics_configuration

  • [ ] get_bucket_notification

  • [X] get_bucket_notification_configuration

  • [X] get_bucket_ownership_controls

  • [X] get_bucket_policy

  • [ ] get_bucket_policy_status

  • [X] get_bucket_replication

  • [ ] get_bucket_request_payment

  • [X] get_bucket_tagging

  • [X] get_bucket_versioning

  • [ ] get_bucket_website

  • [X] get_object

  • [X] get_object_acl

  • [X] get_object_attributes

The following attributes are not yet returned: DeleteMarker, RequestCharged, ObjectParts

  • [X] get_object_legal_hold

  • [X] get_object_lock_configuration

  • [ ] get_object_retention

  • [X] get_object_tagging

  • [ ] get_object_torrent

  • [X] get_public_access_block

  • [X] head_bucket

  • [X] head_object

  • [ ] list_bucket_analytics_configurations

  • [ ] list_bucket_intelligent_tiering_configurations

  • [X] list_bucket_inventory_configurations

  • [ ] list_bucket_metrics_configurations

  • [X] list_buckets

  • [ ] list_directory_buckets

  • [X] list_multipart_uploads

The delimiter and max-uploads parameters have not yet been implemented.

  • [X] list_object_versions

The default value for the MaxKeys-argument is 100. This can be configured with an environment variable:

MOTO_S3_DEFAULT_MAX_KEYS=5

  • [X] list_objects

The default value for the MaxKeys-argument is 100. This can be configured with an environment variable:

MOTO_S3_DEFAULT_MAX_KEYS=5

  • [X] list_objects_v2

The default value for the MaxKeys-argument is 100. This can be configured with an environment variable:

MOTO_S3_DEFAULT_MAX_KEYS=5

  • [X] list_parts

  • [ ] put_bucket_abac

  • [X] put_bucket_accelerate_configuration

  • [X] put_bucket_acl

  • [ ] put_bucket_analytics_configuration

  • [X] put_bucket_cors

Note that the moto server configures global wildcard CORS settings by default. To avoid this from overriding empty bucket CORS, disable global CORS with an environment variable:

MOTO_DISABLE_GLOBAL_CORS=true

  • [X] put_bucket_encryption

  • [ ] put_bucket_intelligent_tiering_configuration

  • [X] put_bucket_inventory_configuration

  • [X] put_bucket_lifecycle

  • [X] put_bucket_lifecycle_configuration

  • [X] put_bucket_logging

  • [ ] put_bucket_metrics_configuration

  • [ ] put_bucket_notification

  • [X] put_bucket_notification_configuration

The configuration can be persisted, but at the moment we only send notifications to the following targets:

  • AWSLambda

  • SNS

  • SQS

  • EventBridge

For the following events:

  • ‘s3:ObjectCreated:CompleteMultipartUpload’

  • ‘s3:ObjectCreated:Copy’

  • ‘s3:ObjectCreated:Post’

  • ‘s3:ObjectCreated:Put’

  • ‘s3:ObjectDeleted’

  • ‘s3:ObjectRestore:Post’

  • [X] put_bucket_ownership_controls

  • [X] put_bucket_policy

Basic policy enforcement is in place.

Restrictions:

  • Only statements with principal=* are taken into account

  • Conditions are not taken into account

  • [X] put_bucket_replication

  • [ ] put_bucket_request_payment

  • [X] put_bucket_tagging

  • [X] put_bucket_versioning

  • [X] put_bucket_website

  • [X] put_object

  • [X] put_object_acl

  • [X] put_object_legal_hold

  • [X] put_object_lock_configuration

  • [X] put_object_retention

  • [X] put_object_tagging

  • [X] put_public_access_block

  • [ ] rename_object

  • [X] restore_object

  • [X] select_object_content

Highly experimental. Please raise an issue if you find any inconsistencies/bugs.

Known missing features:

  • Function aliases (count(*) as cnt)

  • Most functions (only count() is supported)

  • Result is always in JSON

  • FieldDelimiters are ignored

  • [ ] update_bucket_metadata_inventory_table_configuration

  • [ ] update_bucket_metadata_journal_table_configuration

  • [ ] update_object_encryption

  • [X] upload_file

  • [X] upload_fileobj

  • [X] upload_part

  • [X] upload_part_copy

  • [ ] write_get_object_response