Actually the post explains the motivation well. The basic idea is to add capabilities to redis at native speeds, without having to either convince antirez that they're necessary or forking redis.
I work for Redis Labs, and over the course of the past few weeks we've experimented with some modules using the API.
Some of the things we did that might answer some of your questions:
a PAM based authentication module, which IIRC can plug into LDAP potentially.
A bloom filter implementation on top of bitmaps.
An inverted index that doesn't use existing data types but rather creates compressed indexes directly using redis strings (I wrote that one personally BTW)
An API to store images on redis and retrieve them resized or cropped.
So right now the only way to create new data types is to piggyback strings as raw memory, but the final version of the API will include actually registering new data types.
Maybe you can relay the information to the appropriate person, but on http://redismodules.com, I can neither right click "open in new tab" or ctrl+click to open in a new tab.
This breaks my normal browsing habit of opening new tabs as I see something that interests me, but waiting until I finish skimming the list to read.
> a PAM based authentication module, which IIRC can plug into LDAP potentially.
Yes, pam-ldap / pam-sssd / pam-winbind plug into various LDAP implementations (plain LDAP / ActiveDirectory) and work transparently with all PAM clients.
What I meant by multi-tenant is a multi-user redis (just like multi-user mysql/postgresql. (ie) i can create users and have them use redis completely separately on a single instances on redis.
There's been a lot of unusual features I've been hoping Redis will support but obviously given their edge-case nature it's unlikely that they'd ever be implemented.
This is great to see. Instead of cluttering up the core engine with junk people can mix in their own functionality.
This looks interesting. I've only glanced at the docs, but it looks like this module system lets you define new commands but not necessarily new data structures (since you'd have to implement RDB/AOF serialization for those).
Are there plans for that? It's probably possible to do most of that with this module system but you'd have to serialize the data structure into a string and "type X" wouldn't return your custom data structure type.
He mentioned in the keynote being able to do your own datastructures by raw access to the data and primitives like seeks and next/prev .
Sounds like it's at least planned, if not roughed in.
This is great and long awaited, please consider some kind of a namespacing scheme for module-defined commands though, or things can easily become messy.
In the module example the "hello." prefix is specified for all defined commands - it would probably be better to specify it once on module initialization and have all commands defined by the module prefixed with it automatically, the module registries could then manage the prefix registration/allocation to module authors.
There are proprietary modules for linux, and they are distributed and used, but there isn't really a full consensus on whether they are legal. You can certainly do it and get away with it though.
I've written something similar too. Adding plugins/modules was like a rite of passage for anyone digging into the redis source. I'm definitely happy to see modules getting official support now though.
If these modules are going to be as fast as the native commands, is there any reason to not lift the entire native command set out into a set of "base" modules?
I get some of my best work done on a plane. I flew out of LA a few months back and a family of five (mom, dad, daughter, and two young boys) had the rows behind me. They made a lot of noise and were just being kids. The dad apologized and offered to buy me a drink. I told him no need, because that activity allowed me to focus on what I was doing. I think that I added a new feature that I was mulling over during that flight.
I love coding in public spaces. Life acts as a very good white noise
I love working on the plane too. Noise-cancelling headphones and a 13" MacBook Pro. I usually code for around 40% of the SF to London flight. Great opportunity for uninterrupted flow.
This is how I try to justify flying up front (if the price works out right). I take the extra cost and attempt to "pay it back" by how many hours of extra work I can get done (or, if an overnight flight, extra work I can do the next day after sleeping the whole way instead of feeling like a zombie).
One of the companies I started a few years back was born in the back of the highway 17 Express, a bus with cramped seats that blazes down a really windy road at 60 mph. It even has wifi (for some parts of the ride). After that, working on airplanes was easy. You just need to fight a bit for elbow room and hope you aren't seated between two obese people. If you're really lucky, the person in front of you will be too concerned with others to lean their chair back...but even if they do, it's workable.
It's not the constant interruptions (headphones do a good job of dealing with that), its the lack of internet access that is the showstopper - I don't have all the APIs + docs in my head.
I use https://kapeli.com/dash for offline documentation for when I'm on flights. Also, WiFi has been available on more and more flights. I actually have found long flights good for solid focused coding sessions (especially when I already have thought through what I want to build).
edit: woops, looks like someone submitted this at the same time. In any case, working offline also forces you to reign in your app dependencies.
@antirez, is Redis 4.0 going to be part of the more rapid release schedule (will we see it in 3, 6 months) or are major upgrades still annual releases (we'll see 4.0 in 12 months)?
Definitely part of the short release cycle, we'll probably focus on getting modules done and the features already pending into "unstable", and a few more minor stuff, and release it.
Shows some sample modules from Redis Labs called RedisEx, pretty useful stuff (although most of these should really be in Redis by now but at least it's an option to include as modules).
This exactly the point of the module system. You don't need to either convince antirez to accept/implement a new feature, or fork redis if he doesn't agree. It will make the dynamics of redis much more interesting IMO
databases (redis included) are awesome. It's always surprising to me that we don't run web apps directly on the database; they're high-performance, durable and parallel, and for some apps are already the perf bottleneck.
I think the only reason we don't is that deployment and testing isn't straightforward. Fingers crossed antirez will get this right.
In an ops-type capacity, I support developers. I keep up with news like this specifically to avoid winding up with modules somebody else trusts, but didn't inspect.
I tried to read the article but I couldn't. It's all one big <pre> block. The word wrapping is wrong on input! Individual paragraphs are groups of individual lines and extra long lines that the browser wraps for you. So you end up with lines in the middle of paragraphs that only have one or two words.
Net result is it's very hard to read.
I tried Safari's reader mode, but that was even worse. Because all formatting comes from happenstance in the <pre> tag reader mode loses all formatting. The entire article (and code!) becomes one long block of text.
I'm continually amazed at people's attempts to reinvent text layout in the browser.
I make these kind of comments every once in a while. They're usually downvoted, I assume, for being off-topic or unnecessarily harsh ('We might have cured cancer!' vs 'I can't read the article because of the video behind the text.').
But I hope some people see them and work on the issue.
Sometimes, if enough other people agree, these kind of comments get voted up. I guess this issue didn't annoy most people enough to get the up votes.
What kind of modules did you have in mind when designing this? Are any of the below possible?
* ldap backend login system
* multi-tenant redis
* alternate db format/file system.
* implement a new redis data type. say built-in date support
* can i implement new commands and operators on said data types
An of course, thanks for this fantastic software. We use it everyday. Will read in more detail about this over the weekend :-)