Data storage software: How to make the most of Redis?

Data storage software: How to make the most of Redis?

One of our programmers recently worked with Redis, a data storage software that does not use the classic table and column format, but allows to use and store several types of data and access it with a Key.

Redis, which in technical jargon is a Key-Value Storage, supports data lists, data sets –which do not allow duplicate data–, ordered sets –that allow to give a score to a certain data, and use that score to perform various operations– and even simple structures called Hashes, among other things.

The first thing you discover when starting to work with this type of non-traditionally- structured databases is that the NAME of the Key is critical, and that it is important to know which operations are high-speed operations, that is, O(1). This parameter is called Time Complexity, and each one of the Redis commands indicates that factor, for example, GET or SET, which are O(1), while SINTER –which allows to create an intersection of two sets– is O(N*M), where N is the amount of elements of the smallest set and M is the amount of sets involved in the Operation. Therefore, we can evaluate how well-thought a software that uses Redis is by analyzing the distribution of time complexity use, wishing O(1) for 100%.

Another very useful feature of Redis is to be able to give an expiration date to a certain Key. For example, we could build a session management software using a key name such as SESSION_ {UserId} that will store the session id value. And we could set that key to expire in 10 minutes. This concept of volatility is very well implemented in Redis. For further information and study, check here.

Our programmer, Arturo, needed to implement expiration but at the data level, and not at the Key level. Today we want to share with you a bit of the code we implemented, since there are no examples on the web for other languages and nothing for Python. The code is a class called DeadSet (in honor of both the implemented functionality, expiration of items in Redis sets, but also of a professional player of the game Diablo III of Blizzard company of which our programmer is a fan). Let’s see an example of use:

import redis
import datetime
from deadset import DeadSet

r = redis.StrictRedis(...)
expDate = datetime.timedelta(days=7)
dsetkey = "DEAD_IN_A_WEEK"

datakey = "THIS_HOLDS_DATA"
data = {"a": 4, "b": 2}
dset = DeadSet(keyname=dsetkey,
               default_expire=expDate,
               rdb=r)
dset.add_expiring_item(itemId=datakey, itemdata=data)

In the example, we work on the key DEAD_IN_A_WEEK, to which an item is added with the name “THIS_HOLDS_DATA”, which in turn will become a Redis Hash type key that will store the information of the variable data (a=4, b=2). Both the item and the key will expire automatically within seven days because we can set the expiration through a python timedelta. This means that we can convert O(N) operations that are costly in terms of time, into immediate O(1) operations, with automatic expiration as an added value, thus optimizing our software. You can access the DeadSet class in our git repository here.

Fraud in online payment systems: 
your personal information is worth more than you can imagine

Fraud in online payment systems: 
your personal information is worth more than you can imagine

Open Sesame! The uses of biometrics as applied to InfoSec

Open Sesame! The uses of biometrics as applied to InfoSec