via tweetie
MySQL Observer concept
I have been playing with Redis, MongoDB and CouchDB, in my exploration of the burgeoning NoSQL movement.
Having used MySQL for years, I have used most of the features, short of clustering and stored procedures. MySQL has a LOT of great features, but most require careful planning to exploit fully.
Briefly I was enamored with fulltext search, but it tends to cause issues as your database grows and you need to make changes to the schema.
As I thought about replicating data, in the context of MySQL and the new NoSQL kids, an idea took hold - the MySQL Observer.
What if MySQL could replicate data to a different kind of server? One with different indexes?
For example, in a typical master - slave replication deployment, servers replicate the data, structure and indexes.
If you want a few servers optimized for search or reporting, its not possible today.
In steps the MySQL Observer, which is a MySQL server configured to observe the replication stream from any slave or master server. The difference being, it would only capture structure and data, but could overlay its own indexes.
The Observer could never replicate or be elevated to master, and significant schema changes would require the indexes be rebuilt.
Your data could be used in the all sorts of ways, while relying on the MySQL engine.
This lead me to another thought, we need a uniform standard for replicating data between data stores. But thats another post...
Simple authentication for Sinatra in Rails Metal
Using Sinatra:Base to build Rails Metal is a joy and really speeds up the process.
I needed to protect API access with simple authentication and after trying a few routes I found this technique:
before do
if request.env['PATH_INFO'].include?('/api/cool_stuff')
Rack::Auth::Basic.new(request, 'API Access') do |username, password|
Account.authenticate(username, password)
end
end
end
The trick was keeping the authentication from triggering for requests not meant for this path.
Advantages of taking the path least likely..
This weekend I worked on a service concept I brainstormed with a client, hoping to spring it on them next week. Since its a prototype I decided to put tools to work I have been playing with, namely Sinatra and Redis.
Late last night during a coding session, I was struck by how refreshing it felt to NOT be constrained by the Rails framework.
Being a conceptual developer, I tend to bypass some steps during the initial creation of new applications. Experience has taught me it can be burdensome to try to write tests or specs for things I didn't even know would work.
After all, a painter doesn't test for color accuracy while mixing the paints, that happens when it is forged into a commercial product by the printers, right?
Anyway, last night I didn't have any generators, templates or scaffolds to jump start the project, and building without ActiveRecord and SQL on the back-end I had to craft my own models to interact with Redis.
This made me truly appreciate writing specs early and often, probably a couple hours too late, but still I did appreciate and find joy in it.
Jailit for fun!
On a couple of projects I have run into issue with needing to secure web service from abuse and fraud. One client is building a sophisticated security system, but I see the need for simple web service. Over the weekend, I whipped up JailIt in a few hours. Its fast and straightforward, basically a giant hash table. For the REST API I am using Sinatra, storage is handled by Redis and in front of everything is Nginx with Passenger. Before I let the masses in, I have to flesh out the account security, data partitioning and monitoring. From a design standpoint, the whole thing runs in under 70MB of memory on a sweet slice of Ubuntu served by Slicehost.
March update
Whatever happened to project March? Well, I have built a prototype and am working on the user experience, which is quite different from what was originally envisioned. March piggybacks on top of existing workflows and protocols, sort of insinuating itself into everyday work without being obtrusive. Some elements of March are available in high-end products and can be cobbled together, sort of, using pieces of everyday tools. But without unified management or reporting mechanisms for managers and team members to use. Why haven't I pushed it out the door yet? I have talked through all the usability scenarios with friends and colleagues, but need to do a bit of real world testing. So, next month I will be putting it to the test with a client. If it works as expected, I will launch sometime in August or September. Development took a radical turn recently. Originally March was built using Rails, MySQL and a commercial server product that shall remain unnamed. In the last couple weeks, I have been experimenting with a new stack that consists of Ruby (Sinatra and EventMachine), Redis, Nginx and good old fashioned NFS. Sometime next week I should know if this new stack performs as envisioned. If so, it should scale a mile wide and a few inches (rack space?) deep. Don't think its uber cool. It just sprinkles productivity boosts in the right places.
Kickass Review is coming
Working on diverse projects and having a voracious appetite for new technology, clients and friends are always turning to me for my take on how they can best use technology to their businesses. I kicked the idea around for a few weeks and decided that a monthly email with my experiences, experiments and tips could be fun. It should help me refine my own philosophy on these topics - writing them down tends to spur that growth. So, look for an announcement soon when I through up the subscription list. It will be free and creative commons licensed. Have a great memorial day weekend!
No way out of outlook?
Do more powerful tools with more options really make you more productive? Dealing with a wide range of people using every type of email, I have noticed a trend. Those with Outlook seem to be the slowest to
