SyntaxHighlighter

SyntaxHighlighter

Showing posts with label aws. Show all posts
Showing posts with label aws. Show all posts

Wednesday, August 30, 2017

Serverless Tip: Use the "artifact" Directive to Deploy Your Pre-Built Lambda Zip File

tl;dr: you can deploy pre-built zip files (e.g. for your Python Lambda) using the "artifact" directive in the serverless framework.

AWS Lambda is Great!

I've been doing a lot of work recently with AWS Lambda. And I'm a fan. The combination of API Gateway + Lambda + Python, together with other AWS services including DynamoDB and S3, not to mention the awesome array of Python open source libraries, means I'm churning out all sorts of microservices with glee.

The serverless paradigm is quite different than the traditional (serverfull?) paradigm. As well as adjusting the architectural style to take advantage of what Lambda offers, deploying the code and all of its dependencies is quite different. After looking at some alternatives, we concluded that the Serverless Framework best fit our requirements.

The Serverless Framework is Great!

Rather than crafting complex CloudFormation configurations to manage my microservices in AWS, I use the Serverless Framework. (The framework also works with Apache OpenWhisk, Microsoft Azure and Google Cloud). Essentially Serverless is a simpler CloudFormation, specific to Lambda-centric deployments. (To be clear, it doesn't just help with deployments of AWS Lambda - Serverless covers a wide and growing range of AWS services).

Mainly by studying (*cough* copy-n-pasting *cough*) the extensive range of examples, and sometimes resorting to actually reading the manual, I've been able to get even quite complex setups to work, with fairly simple YAML configuration files. So, I recommend Serverless. (Although AWS themselves are developing an eerily similar alternative, in SAM, which you may also want to check out).

AWS Lambda has Limits

Sometimes you need to do stuff outside the Serverless Framework, but you still want to use all the other cool stuff it does for you.

For example, AWS Lambda has certain limits. This includes a 50Mbyte deployment limit per Lambda. Now, Serverless does let you control what goes into the Lambda package via the "include" and "exclude" directives, within the "package" directive. But, sometimes, you're sailing very close to the 50Mb limit and the only way to stay underneath is to directly create your zip package yourself. Or, in my case, you have a zip file which has precisely what you need, but you also need to manipulate it to add in a pickled bit of code. (Which you do via the Python zipfile library).


It took me a while to figure out, but you can use the "artifact" directive as the way to deploy a zip you've packaged already.

So, there you have it: Lambda is great, but you should use Serverless (or something like it) to simplify your deployments.  And you can deploy pre-built zip files using the "artifact" directive.

Tuesday, February 21, 2017

Serverless LinkLog #1

This is the second in an occasional series of LinkLog posts. I list links to interesting articles, video and audio items on "serverless computing" and allied topics. Just like in my Machine Learning LinkLog #1 I group the items into three broad categories:

Introductory Non Technical - Aimed at the general reader or, perhaps, technical manager who wants to learn about Serverless, but is not aiming to be a practitioner
Introductory Technical - Aimed at someone who is comfortable with programming and technology, but wishes to learn how to work with serverless tools or techniques
In Depth Technical - Aimed at someone who is comfortable with the fundamentals of serverless technology, but wants to learn more about a particular aspect or wants to master "day two" problems.

I also intend to indicate whether the item is (mainly) a slidedeck, a video, a single article or a series of items. A secondary goal is to avoid the trope of using pictures of clouds to illustrate serverless topics - however, I apologize in advance if I fail.

If You Only Read One Thing About Serverless

Having said all of that, if you only want to read one thing about serverless, then I recommend "Serverless Archictectures" by Mike Roberts. It will take you from 0 to 60 - not only understanding what are serverless architectures and how they relate to Backend-As-A-Service and Function-As-A-Service, but also their future potential and current shortcomings. This is for both technical and non technical people and will give you much greater insight than any of the introductory material listed below. However, it is quite long - I wound up reading it in chunks. So, you might read one or more of the "intro" items, before coming back to this one.

Introductory Non Technical

How Can Enterprises Leverage Serverless Computing Platforms?
Two page introduction from Forbes, talks a lot about AWS Lambda, but mentions competitors. Discusses how various large enterprises are blending serverless with their other compute strategies.

A Guide to Serverless Computing with AWS Lambda
Decent introduction for technical managers - has some source code, but it really isn't aimed at help you write lambda code. Very AWS specific, of course, but has a nod to competitor offerings.

AWS Lambda is Everywhere
Adrian Cockcroft gives an overview of AWS Lambda, with particular emphasis on the latest (November 2016) announcements from AWS re:Invent.

Evolution of Business Logic from Monoliths through Microservices, to Functions

Adrian Cockcroft again - portraying AWS Lambda as the logical next step in the business of computing.

WTF IS OPERATIONS? #SERVERLESS
OPERATIONAL BEST PRACTICES #SERVERLESS
Two part article from Charity Majors. Pours cold water on the notion that "serverless == noops" i.e. that just because you are not provisioning servers yourself that you therefore don't need to worry about operational problems anymore. In fact, she makes a compelling case that serverless requires all developers to think much more profoundly about operational, "day two" problems. She also points out how much more opaque serverless applications are, making them much harder to debug.


Introductory Technical

Serverless
This is a bit confusing but ... there is an open source framework called "Serverless" aimed at making it easier to work with various technologies - including AWS Lambda, Azure Functions and Google CloudFunctions. There are other frameworks (see Zappa and Apex below) but this one seems popular (maybe because it is likely to come up when you google for "serverlesss"? Or, more seriously, maybe because it supports a variety of technologies and deployment scenarios - including local for development)
http://www.serverless.com

Zappa
Open source framework for for turning Python WSGI applications into serverless apps using AWS Lambda and Gateway. Supports advnaced capabilities, including scheduling and keep_warm (for better performance).

Apex
Open source framework aimed at making AWS Lambda easier to work with. Adds support for non Lambda-native languages (such as golang) via a node.js shim.

Writing a cron job microservice with Serverless and AWS Lambda
A hands on article takes you through a practical example of how to write a cron-equivalent using Serverless (the framework) and AWS Lambda.

How to build powerful back-ends easily with Serverless
An image processing server in node,js with the Serverless framework, using a bundle of AWS services - Lambda, Rekognition and S3.

Bring static to life using serverless

Serverless Video Playlist
Nine video playlist of hands-on basic coding with AWS serverless with an emphasis on Lambda and node.js.

In Depth Technical

Serverless Workflows on AWS: My Journey From SWF to Step Functions
In depth technical discussion of AWS Simple Workflow and Lambda, with an emphasis on AWS Step Functions - sort of like Finite State Machines.

Fission: Serverless Functions as a Service for Kubernetes
Using Kubernetes to host Functions as a Service. Kubernetes is a way to host Linux containers within a cluster. So, this is a way to achieve a serverless architecture without using any of Lambda, Azure or CloudFunctions. In other words, this is a kind of "private serverless" (akin to "private cloud").

StreamAlert
Airbnb's open source framework for analyzing massive log streams and alerting on them by defining rules in Python. Uses AWS Lambda and Kinesis, amongst other technologies.

Monday, July 22, 2013

AWS, SQS and Poisonous Items

I've been using Amazon Web Services to handle large scale processing of content. One handy AWS is SQS, the Simple Queue Service. This is great, since it allows you to decouple (and hence scale) your processing. (There are other advantages besides). However, I've encountered a problem with queues that I've nicknamed "The Poisonous Item". I thought I would share it, together with a couple of workarounds (but not solutions) for handling it.
Q is for Queue by Darren Tunnicliff
http://www.flickr.com/photos/darrentunnicliff/3717976312/

A Typical Architecture Using SQS
Let's pretend that you have a system that uses SQS to process XML documents and update them to your Amazon S3 storage. The processing of each XML document can take a variable amount of time (e.g. depending on the size or complexity of the document), so you decide you want to make it scale nicely. You therefore create an application that writes the document ids to an SQS queue and then you create a second application that performs the processing on each document and writes the result to S3. Because each document can be handled independently from all of the others, you can therefore run as many instances of the document processing application as you like, in parallel, fed by your SQS queue. This can be pictured as in the diagram below.
Visibility Timeout
The SQS queue has a visibility timeout, with a default of 30 seconds. What this means is that when a service fetches something from the queue, the item is hidden for a period of time, to allow for the service to handle it. If all goes well, then the service deletes the original item from the queue. However, if the service crashes, the item becomes visible on the queue again (because the visibility timeout expires). This is all a good thing, since it means that your service is reliable in the face of problems (like an instance of your XML processing application crashing).
Poison by Thorius
http://www.flickr.com/photos/thorius/288024760/
Poison Items
The “poison” item scenario is as follows: there’s some problem with one particular item, e.g. it is really huge and takes, say, 10 minutes to process. That means that the item will timeout and become available to each of the consuming services in turn, whilst the others are still processing it. (I also call this the “Titanic” effect where a safety measure actually makes something more vulnerable to certain issues).

The problem, of course, is that every single instance of your XML processing application will eventually be "poisoned" by the long-running item. In the best case, one of the applications eventually completes and removes the poison item from the SQS queue. Even in this case, however, your applications are doing a lot of duplicate work.

So, how can you try to cope with this?
I need a timeout by Ruth Tsang
http://www.flickr.com/photos/ruthtsang/7247429542/
A Longer Timeout
The first workaround, of course, is to bump up the visibility timeout to be higher than the default 30 seconds. This is relatively simple to do and can completely eliminate the poison item problem altogether. Exactly what level to pick for your timeout is, of course, very much dependent on your application (if it is very high - in the hours - then maybe you need to break your processing into smaller steps?) But you still need to have a timeout, to cope with the legitimate problem of a crashed system.

One rule of thumb is to estimate two standard deviations for the range of processing times. That way, if your processing times conform to a "bell curve" (more formally, a normal distribution) then your timeout will cover almost 98% of the situations your application will encounter. But you also need to weigh this against having too many items in the queue be invisible, since it might mislead you into thinking your processing is complete, when it isn't. Or, if you're using autoscaling, it might result in winding down servers too quickly (since legitimate items might be invisible too long).
Jude'll Fix It no. 103 by Derek Davalos
http://www.flickr.com/photos/derekdavalos/9203747318/
Fix It!
The other workaround is to try to "fix" the reason for the lengthy processing time. Of course, this is extremely dependent on why the times are variable in the first place. In my situation, my XML application assembles smaller documents into larger documents and runs them through an XSLT. Since the number of subdocuments can vary considerably, the processing time varies just as much (if not more so). So, my "fix" was to limit the total number of subdocuments with a reasonable upper limit.  This kind of limit might not work for you (and is certainly a workaround).

Potatoes-Kipfler-HeatAffectedHarvest-928 8-2040gram

Potatoes-Kipfler-Heat affected harvest 2040gram by graibeard
http://www.flickr.com/photos/graibeard/4121218392/
Suggestions?
What else can you do to work around the "poison item" problem?

Thursday, January 24, 2013

I have been playing a lot with Amazon Web Services. For numerous reasons, I principally like to use these key bits of software in the work I do:

Python
lxml
s3cmd

As a consequence, I find myself repeatedly doing the following steps to bring the Amazon Linux up to scratch for what I need. I thought I would document them here, in case anyone else finds these steps useful. But also as an easy way for me to find it again...

To Just Generally Bring the Server Up To Date
sudo yum update

Amongst other things, this brings you to Python 2.6, which is sufficiently up-to-date for what I need. (By the time you or future me reads this, I suppose it might be more up-to-date than that).

Install lxml
lxml is the best library I've found for working with XML in Python. It is compatible with, but offers lots of nice enhancements beyond, the standard elementtree, including better support for XPath and built in support for XSLT processing.

Based on this very handy blog post, I do the following to install lxml

sudo yum install gcc
sudo yum install python26-devel
sudo yum install libxslt
sudo yum install libxslt-devel
sudo yum install libxml2-devel
sudo easy_install libxml


That last step to install libxml can take a few minutes. But the whole thing typically takes perhaps ten minutes.



Install s3cmd
Since I do a lot of work with Amazon s3, it is handy to have a command line interface to list, get and put files to s3 buckets. I tried following the instructions about how to install s3cmd on the site, but it just wouldn't work. So, now I do this and it works like a charm:

sudo yum --enablerepo epel install s3cmd


And you can run s3cmd --configure to set up and test out the s3 configuration, if you like.