Commit graph

347 commits

Author SHA1 Message Date
jweite
49d92861c0
Iam cloudformation update, singificant cloudformation refactoring (#3218)
* IAM User Cloudformation Enhancements: update, delete, getatt.

* AWS::IAM::Policy Support

* Added unit tests for AWS:IAM:Policy for roles and groups.  Fixed bug related to groups.

* AWS:IAM:AccessKey CloudFormation support.

* Refactor of CloudFormation parsing.py methods to simplify and standardize how they call to the models.  Adjusted some models accordingly.

* Further model CloudFormation support changes to align with revised CloudFormation logic.  Mostly avoidance of getting resoure name from properties.

* Support for Kinesis Stream RetentionPeriodHours param.

* Kinesis Stream Cloudformation Tag Support.

* Added omitted 'region' param to boto3.client() calls in new tests.

Co-authored-by: Joseph Weitekamp <jweite@amazon.com>
2020-08-27 10:11:47 +01:00
jweite
3342d49a43
S3 cloudformation update (#3199)
* First cut of S3 Cloudformation Update support: encryption property.

* Update type support for S3.  Abstract base class for CloudFormation-aware models, as designed by @bblommers, introduced to decentralize CloudFormation resource and name property values to model objects.

* Blackened...

* Un-renamed param in s3.models.update_from_cloudformation_json() and its call to stay compatible with other modules.

Co-authored-by: Bert Blommers <bblommers@users.noreply.github.com>
Co-authored-by: Joseph Weitekamp <jweite@amazon.com>
Co-authored-by: Bert Blommers <info@bertblommers.nl>
2020-08-01 22:43:03 +01:00
Adam Richie-Halford
9a9a1d8413
Decentralize cloudformation naming responsibilities (#3201)
* #3127 - Decentralize CF naming responsibilities

* Decentralize CloudFormation naming responsibilities

* Update URLs in cloudformation_resource_type functions

* Fix flake8 errors

* Black formatting

* Add a bunch of imports to populate CloudFormationModel.__subclasses__

* Add noqa to s3 models import statement in cloudformation/parsing.py

* Black formatting

* Remove debugging print statement

Co-authored-by: Bert Blommers <info@bertblommers.nl>
2020-08-01 15:23:36 +01:00
Steve Pulec
980525c3dd
Merge pull request #3121 from bblommers/bugfix/2104
S3 - Persist metadata for Presigned URL
2020-07-26 15:29:38 -05:00
Macwan Nevil
c166d97a97
Bugfix: S3 time precision issue fixed (#3182)
* Bugfix: S3 time precision issue fixed

* Bugfix: S3 time precision issue fixed

* s3 timeformat fix

* Quickfix S3 timefix
2020-07-26 16:00:15 +01:00
Dean Kleissas
936c7c80f1
When generating multi-part upload IDs remove / characters (#3164)
* / are not allowed in upload ids

* ran black
2020-07-22 17:01:31 +01:00
Bert Blommers
8b63421321 S3 - Only add metadata once; use binary file content 2020-07-12 18:29:10 +01:00
Bert Blommers
b33c5dff06 #2104 - S3 - Persist metadata for Presigned URL 2020-07-12 13:33:46 +01:00
Bert Blommers
9ed7ba58df S3 - Implement delete_object_tagging 2020-06-20 12:15:29 +01:00
Dawn James
8ce12027dd
Return correct error when creating a bucket with empty CreateBucketConfiguration (#3079)
* Several updates to the contributor documentation with extra information.

* Fix failing test by providing a region.

* Create test for issue 2210.

* Check if CreateBucketConfiguration is supplied and empty; raise MalformedXML error if so.
2020-06-19 11:44:43 +01:00
Steve Pulec
80a3571ff4
Merge pull request #3040 from timvlaer/fix/s3_bucket_creation_date_tz
Make sure the UTC tz is included in the bucket creation timestamp
2020-06-11 20:59:24 -05:00
Bert Blommers
20784a2d67 Improve implementation coverage 2020-06-06 13:15:50 +01:00
Tim Van Laer
ca49b41599 Make sure the UTC tz is included in the bucket creation timestamp 2020-06-02 11:32:47 +02:00
jweite
4d3e3c8c5e
implemented s3 default encryption methods (#3022)
* implemented s3 default encryption methods

* PR adjustments: moved logic for retrieving bucket's encrypted status to the backend.

Co-authored-by: Joseph Weitekamp <jweite@amazon.com>
2020-05-27 17:21:03 +01:00
usmangani1
774a764b69
Fix s3 Added Error handling in case of invalid uploadID (#2979)
* Added Error handling in case of invalid uploadID

* Linting

* added assertions

* Linting

Co-authored-by: usmankb <usman@krazybee.com>
Co-authored-by: Bert Blommers <info@bertblommers.nl>
2020-05-12 14:59:07 +01:00
Bert Blommers
49b00942c3 S3 DeleteObjects - Allow multiple querystring formats 2020-05-01 13:50:11 +01:00
Michael Penkov
45d99aef69 fix list comprehension 2020-04-29 13:03:03 +09:00
Mike Grima
dd22e7855a Fixed a regression with CloudWatch 2020-04-27 12:56:17 -07:00
Bert Blommers
b5a894dd89
Merge pull request #2919 from antoinewdg-mt/fix-s3-delete-objects-with-versioning
Handle VersionId in S3:delete_objects
2020-04-27 15:38:06 +01:00
Antoine Wendlinger
41abd4344b Use xmltodict for parsing 2020-04-27 11:42:27 +02:00
Steve Pulec
efbb9c54a5
Merge pull request #2909 from oliparcol/bug/2759
S3 - Allow keys to start with leading slashes
2020-04-25 18:46:25 -05:00
Steve Pulec
b24b7cb858
Merge branch 'master' into feature/extend_generic_tagger_to_s3 2020-04-25 18:40:50 -05:00
Steve Pulec
f4338876ff
Merge pull request #2834 from bblommers/feature/cloudwatch-integrated-metrics
Cloudwatch  - Add default metrics for other services
2020-04-25 18:35:17 -05:00
Olivier Parent Colombel
d852f7dd06 Fixing lint errors. 2020-04-25 17:49:50 +02:00
Olivier Parent Colombel
637e0188a2 Allow S3 keys to start with leading slashes. 2020-04-25 17:49:50 +02:00
Antoine Wendlinger
9ed6e52d0a Handle VersionId in S3:delete_objects
VersionId is not read in delete_objects requests, and the behavior
differs from its singular counterpart delete_object.

This fixes the issue.
2020-04-23 19:01:58 +02:00
Daniel Wallace
50a147592d
Make all CallbackResponse requests into a Werkzeug Request
The "request" object in CallbackResponse is the PreparedRequest send by
whatever client is used to contact the mocked moto service. This can end
up with unparsed bodies, as we added for processing presigned post
requests in #2155. This will make sure that all of the requests comming
in from mocked functions also get processed by werkzeug as if it was
running a live server.
2020-04-22 09:08:30 -05:00
Daniel Wallace
80b27a6b93
blacken 2020-04-21 21:43:32 -05:00
Daniel Wallace
4b0ba73204
use werkzeug
hooray, thanks pallets discord!
2020-04-21 20:13:53 -05:00
Daniel Wallace
49b056563a
process multipart form 2020-04-21 19:52:10 -05:00
Daniel Wallace
b3f6e5ab2f
add test 2020-04-21 18:07:50 -05:00
Daniel Wallace
156ba56fdc
set default status for s3 posts
and add support for success_action_redirect.
2020-04-21 18:06:43 -05:00
David Holroyd
a845de1142 PutObject and DeleteObject should produce no XML
S3 itself produces an empty body, with any response metadata in HTTP
headers only.
2020-04-06 21:01:43 +01:00
David Holroyd
16db824d8a Fix response XML structure 2020-04-06 00:19:19 +01:00
Bert Blommers
8dbfd43c5c Use TaggingService for S3 - Cleanup 2020-04-01 15:35:25 +01:00
Bert Blommers
f7ad4cbc09 Use TaggingService for S3 Objects 2020-03-31 12:04:04 +01:00
Bert Blommers
6dd6686afc Use TaggingService for S3 Buckets 2020-03-31 11:10:38 +01:00
Mike Grima
0c191ac33b Raise errors on tagging buckets with aws:*
Cannot tag S3 buckets with reserved tag key space `aws:`
2020-03-30 17:27:06 -07:00
Bert Blommers
c3865532f9 #2711 - Register default S3 metrics in CloudWatch 2020-03-23 15:53:39 +00:00
Steve Pulec
94f85902bc
Merge pull request #2821 from bblommers/feature/2255
CloudFormation - Add Support for FN:Transform and AWS:Include
2020-03-19 19:40:46 -05:00
Steve Pulec
c13aadf75b
Merge pull request #2816 from bblommers/bugfix/2789
S3 - Bugfix - Remove possibilty for concurrent access to file content
2020-03-19 19:37:14 -05:00
Bert Blommers
cbf0397953 #2255 - CF - Implement FN::Transform and AWS::Include 2020-03-18 13:02:07 +00:00
Bert Blommers
5e4736e233 Remove unnecessary print-statements 2020-03-17 09:19:57 +00:00
Bert Blommers
e2434cbf6f Remove unnecessary lock 2020-03-17 09:19:02 +00:00
Bert Blommers
d8423b5de0 Optimize content length for large files 2020-03-17 09:16:12 +00:00
Asher Foa
374b623e1d Fix some 'DeprecationWarning: invalid escape sequence' warnings and use str.format for string interpolation.
I am seeing a lot of deperecation warnings when I use moto for my tests (running under pytest), so I figured I'll clean up some of them.
2020-03-12 13:50:59 -07:00
Bert Blommers
3802767817 S3 - Add test case to showcase bug when downloading large files 2020-03-12 12:25:31 +00:00
Steve Pulec
6b6a059350
Merge pull request #2132 from Projectplace/tagging-directive
Support x-amz-tagging-directive in s3 copy_object
2020-02-17 20:00:30 -06:00
Steve Pulec
a1ffb47ae4
Merge pull request #2751 from mikegrima/s3control
Implemented S3 Account-level public access block.
2020-02-17 19:53:15 -06:00
Mike Grima
11b7be0e85 Implemented S3 Account-level public access block.
- Also added AWS Config listing and fetching support
- Also fixed Lambda test breakage
2020-02-17 15:41:27 -08:00