links for 2010-05-25: MPEG-LA sued; Stonebreaker's VoltDB; Apple playing catch-up?; Android to be "Windows" of phones?

• Company Sues MPEG-LA, Claiming Antitrust Violations Over Patents - Is it just sour grapes from Nero or has MPEG-LA not held up their end of their deal with the DoJ?
  
• Database daddy goes non-relational on NoSQL fanbois - VoltDB, perhaps offering the best of both worlds?
  
• Google vs. Apple in the battle of the fanboys - Is Apple playing catch up?  This years WWDC will tell us for sure.
  
• Is Android destined to be the Windows of smartphones? - Informative piece that highlights Google getting into mobile back in 2005.
  

links for 2010-05-21: Sayonara iPhone; Google and Adobe gang up on Apple

• Sayonara, iPhone: Why I'm Switching to Android - "The Android OS is already outselling iPhone OS in the United States. Now it's blowing past Apple in terms of the technology it's delivering."  "What makes this even more insulting is that Jobs tries to dress up his selfishness as a kind of altruism. He says it's all about creating a beautiful experience, that while he may be selling you an intentionally crippled device, he's doing it for your own good."
  
• Strange Bedfellows, Google And Adobe Gang Up On Apple - It was a bit of a change from day 1 to day 2.
  

links for 2010-05-20: Android on the iPhone; Google IO; Conan visits Google

• VMware’s Cloud Portability Promise Powered By Google - One of the announcements out of Google IO was their working with VMware to offer Java via SpringSource in the cloud.  This is Stefan Ried's analysis.
  
• The Mall of Google - Jeffrey Hammond's analysis of Google IO day 1.
• Conan visits Google - Entertaining if you have the time to watch it.
• How to install Android on your iPhone - I haven't tried it yet, but thinking about it ...

links for 2010-05-18: Oracle releases new version of MySQL Enterprise; Amazon stealing the Cloud; Book on Facebook; Future of Development; Flavors of Cloud Computing

• Oracle Announces Immediate Availability of the Latest Release of MySQL Enterprise - But press release only mentions MySQL Enterprise Monitor 2.2.  No new server yet, but a beta was announced at the MySQL conference.
  
• Amazon is stealing the Cloud - Some interesting stats and survey numbers listed.
• The Age Of Facebook: Excerpts From The New Book By David Kirkpatrick - Interesting insights into how they started and Zuckerberg's unwillingness to cash out.
  
• Understanding the many flavors of Cloud Computing - If you are confused by all the *aaS's.
• Beyond Cassandra: Facebook, Twitter and the Future of Development -  Is the data or code valued more?
  

links for 2010-05-17: Google I/O is the new JavaOne; Government using the Cloud; Caspio = Access + Cloud?; Jobs engages blogger

• Google I/O is the new JavaOne - Google I/O is this week.  Will it become "the" developer conference superseding the new, not at Moscone, JavaOne Oracle puts on in the fall?
• Recovery.gov finds new home in the cloud - It is good to see the government saving the taxpayer a little money by using the cloud.
• Is Caspio The "Microsoft Access" Of The Cloud? - Interesting tool for simple query (and presumably full CRUD) applications.
  
• VMware/SpringSource buys GemStone - I worked with their Smalltalk offerings in the past and really liked them.
• Steve Jobs engages blogger via e-mail - Kudos to Jobs for engaging, but I still think Apple goes too far in "protecting" their users.

Event Processing in the Cloud - Combining Esper with AWS SNS

The folks at Amazon Web Services released their Simple Notification Service in beta a little over a month ago, and I used it to create a loosely coupled weather notification system by publishing weather events to an SNS topic.  It is working great, but I started to think of other types of situations I'd want to monitor and have notifications for, and what better way to process these weather events than with a Complex Event Processing (CEP) engine?  While I could keep extending my shell script, that would be a bit of a hack and I wanted something cleaner that would more easily last the test of time.

Event processing is in many ways a natural extension of messaging infrastructure as the latter is typically used for the passing of events between systems and so having an easy way to plug in or use an event processor makes a lot of sense.  In my case, I wanted to be able to feed periodic weather observations, the raw events, into the messaging infrastructure, then be able to dynamically define the processing rules for those events, and then be able to publish any notifications or "complex events" back into the messaging infrastructure where I could then subscribe or receive them for e-mail/SMS notifications or to kick off other processes or logic.

So, how to put this all together?  There are CEP engines from many of the major middleware vendors, but on my shoestring budget I wasn't about to go spend 5-6 figures on software just to process events for my weather station!  At Sun, as part of the OpenESB project we had developed the Intelligent Event Processor in open-source so it was a natural choice, but it requires the complete ESB infrastructure and can't run standalone.  As I was already using SNS for my messaging and was aiming to make what I created Cloud friendly (see below), I didn't want to bring along an entire ESB, so I needed something else.  Another open-source CEP engine I was familiar with is Esper, and after a quick refresher on its capabilities, I decided it would be perfect.

Esper provides a simple to use but very powerful Java API for configuring the events, queries, patterns, etc. and so I set about creating the interface between SNS and Esper.  What I ended up building has the following capabilities:

• A simple (HTTP/JSON) API for dynamically configuring the events and queries/statements in the engine.
• Receives events from an SNS topic via HTTP and feeds them into Esper.  The publisher of the raw events just publishes to the SNS topic like it was before and I just create a new subscriber for the CEP engine, thus leveraging the beauty of a loosely coupled messaging system.
• Esper processes the events per the configured queries.  As the queries generate results, the results are published to a specified SNS topic which can result in a notification e-mail or kicking off some other process or logic, again leveraging the decoupled nature of SNS.

Now, Esper provides a variety of ways to configure the engine, XML being one of them, but you'll note above I chose to create an HTTP/JSON API to serve as the interface to Esper's Java API for doing this configuration.  This was to be both Cloud friendly and to make the configuration dynamic.  This allows the idea of an event processor in the Cloud that is just available to be configured dynamically, receive events, and publish results, all with no installation or maintenance of the engine or other infrastructure.  One never even needs to log in to a machine to mess with any configuration files.

With this created, I was then able to easily configure the following:

1. I had an existing script called by cron every 10 minutes that sent the weather observation to the Weather Underground.  I added a few lines of code to this script to publish the observation to an SNS topic as well.
2. I created a subscription to the SNS topic that notifies the event processor using HTTP/JSON of each event.
3. I configured my "Cloud CEP" with the following JSON to define the events it would be receiving:

{     "Type": "EventConfiguration",     "Name": "WeatherEvent",     "Fields": [["temp", "double"], ["humidity", "double"], ["dewpoint", "double"]] }

4. I configured a query to use for processing the events, this one telling me the high and low for the past 24 hours every 12 hours at midnight and noon, and sending the result to the specified SNS topic.  The statement is Esper's SQL like syntax for specifying queries:

{     "Type": "ListenerConfiguration",     "Name": "Every 12 hours high and low",     "Statement": "select max(temp) as High, min(temp) as Low from WeatherEvent.win:time(24 hours) output at (1, */12, *, *, *)",     "ActionType": "SNS",     "SNSTopicArn": "arn:aws:sns:us-east-1:444520459559:WeatherEventOutput" }

5. I then created a subscription to the WeatherEventOutput topic to send myself an e-mail.

I now get an e-mail at midnight and noon each day telling me what the high and low were for the past 24 hours.  If I want to know any time the temperature changes more than a certain amount in a given period, or if the temperature shows a trend of getting closer to the dewpoint (i.e. it is about to rain), I can add this to the processing with just a simple HTTP/JSON configuration request.  And with the power of Esper, those scenarios and a lot more are possible for my weather data or any other event data one might have.

The further beauty of this is that while I happen to have this running on my server at home, because of the way it is built using SNS and HTTP, it could be located anywhere on the internet whether hosted on EC2 or your favorite provider, or a "Cloud CEP" service available to all.

Note also that what I've written is by no means tied to SNS either.  It just happens to be Cloud based messaging infrastructure that is convenient to use and gives the benefits of loose coupling.  The input to my Cloud CEP is just an HTTP request and I've written an e-mail and HTTP handler so that generated events can go direct as well instead of going to SNS.

So what do you think?  Do you have scenarios where a Cloud CEP would be useful?  Would you like to try out what I've built thus far?  Feel free to leave a comment or contact me at kschmidt at techrunning dot com.

links for 2010-05-13: Ellison on fixing Sun; Fragmenting Linux a bad thing; Apple's cache waning?; Infinity; SAP acquires Sybase

• Ellison on how he is fixing Sun - He doesn't hold back on pointing out Sun's pre-acquisition flaws.  It is unforgivable if they are all so easily fixed.
• Fragmenting Linux is not the way to beat Apple - But will the vendors using Linux as the core of their mobile strategy align?
  
• Is Apple's cache waning? - The iPhone 4 announcement in June will tell us a lot in answering this question.
• Android's army passes iPhone OS, says NPD - The jump seems pretty big when comparing with other research from 2009, but the trend is clear regardless of the actual numbers.
  
• Infinity - The Hilbert Hotel - If you ever wondered what infinity squared is, this one is for you.
• SAP acquires Sybase - This is a reflection of being a software vs hardware company, but Oracle pays $7.4B for Sun with >$10B in revenue while SAP gets Sybase for $5.4B with about $1.2B in revenue

links for 2010-05-12: Apple update for iPad WiFi woes; Flash not a CPU hog?; Microsoft responds regarding HTML5/H.264; Rich Green CTO at Nokia; Oracle Office; Adaptive PaaS

• Apple Plans Software Update For iPad WiFi Woes - Adjust your screen brightness to solve WiFi issues?
  
• Top Flash Misperceptions : Flash is a CPU Hog - Hopefully the performance issues on Mac are addresed with the API access, but a related issue (to me at least) is that Flash is now used in places it isn't needed so its performance compared to HTML5 is accurate at times.
  
• Microsoft responds to questions about stance on HTML5/H.264 - Not that this will end the debate, but interesting to see Microsoft respond as I don't expect Steve Jobs to issue a followup to his remarks.  Read Ed Bott's analysis.
• Rich Green Nokia's new CTO - I was wondering where he might end up.
• Oracle Office on sale - A lot cheaper than Microsoft Office for sure, but now a perpetual license not a subscription like it was under Sun.
• Adaptive PaaS - Interesting thought on what PaaS needs to do to gain momentum.

links for 2010-05-07: Red Hat vs VMware; Cloud washing; Netflix selects AWS; Android in iPhone 3G

• VMware and Red Hat: The war for the data center - Increasingly overlapping portfolios force them to compete.  VMware's next acquisition SUSE Linux to get an OS?
• May the cloud washing begin: Enterprise software giants graze on cloud startups - "The burden of cloud computing proof rests with the large enterprise software folks. Your job will be to distinguish from mere cloud washing and real IT value."
  
• Netflix Selects Amazon Web Services to Power Mission-Critical Technology Infrastructure - They'd been using AWS for more than a year but are expanding their use.
  
• Android now running on the iPhone 3G - It is still a bit rough (no audio support, but coming in days the story says), but getting close to being able to really run Android on an iPhone 3G