Thursday, 17 September 2015

Redis from C# using a generic linq-based approach

Redis is a powerful cache that acts as a remote in-memory data structure store. It stands for Remote Dictionary Server. Redis allows advanced scenarios and multiple cache servers organized in hierarchies as a master with multiple slaves and supporting persistence. The code below is a generic Redis provider (class that follows the provider pattern) written in C# that makes it easier to use Redis. You will need to add an app setting to your application configuration file (app.config) called "RedisServer" with the hostname of Redis. If a non-standard port is used, just type ":<PORT>" e.g. "someredisserver.cloudapp.net:6784". Of course you need access to a Redis server to make this work. I have tested the C# code below against a Redis Server installed in Ubuntu 14. The Redis version is 2.6.16. The RedisMemoryProvider can be expanded by moving some functionality down to a base class. At the same time, the RedisNativeClient client class gives good support for running low-level operations in Redis. The code you move up a bit to a higher-level code such as RedisMemoryProvider of type T should be concerned around generics, while a non-generic base class can use the Set and Get methods below and so on. The C# console below uses ServiceStack.Redis Nuget package. I have not verified how this technology works against load tests, which will be interesting to write some Load Tests for.

Sample application configuration file:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <add key="RedisServer" value="someredisserver.cloudapp.net" />
  </appSettings>
    <startup> 
        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
    </startup>
</configuration>

RedisMemoryProvider source code:


using ServiceStack.Redis;
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Configuration;
using System.Diagnostics;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace RedisProvider
{
    
    //Based on: http://stackoverflow.com/questions/30818784/generic-object-cache
    public class RedisMemoryProvider<T> where T : class
    {

        private static readonly PooledRedisClientManager m = new PooledRedisClientManager(new string[] { 
            ConfigurationManager.AppSettings["RedisServer"] });

        readonly IDictionary<Type, List<object>> _cache = new ConcurrentDictionary<Type, List<object>>();

        public RedisMemoryProvider()
        {
            LoadIntoCache<T>(); 
        }

        /// <summary>
        /// Load {T} into object cache from Data Store.
        /// </summary>
        /// <typeparam name="T">class</typeparam>
        private void LoadIntoCache<T>() where T : class
        {
            _cache[typeof(T)] = GetAll<T>().Cast<object>().ToList();
        }

        /// <summary>
        /// Find Single {T} in object cache.
        /// </summary>
        /// <typeparam name="T">class</typeparam>
        /// <param name="predicate">linq statement</param>
        /// <returns></returns>
        public T Read(Func<T, bool> predicate)
        {
            List<object> list;
            if (_cache.TryGetValue(typeof(T), out list))
            {
                return list.Cast<T>().Where(predicate).FirstOrDefault(); 
            }
            return null; 
        }

        /// <summary>
        /// Find List<T>(predicate) in cache.
        /// </summary>
        /// <typeparam name="T">class</typeparam>
        /// <param name="predicate">linq statement</param>
        /// <returns></returns>
        public List<T> FindBy<T>(Func<T, bool> predicate) where T : class
        {
            List<object> list;
            if (_cache.TryGetValue(typeof(T), out list))
            {
                return list.Cast<T>().Where(predicate).ToList();
            }
            return new List<T>();
        }

        public T FindById<T>(long id)
        {
            using (var ctx = m.GetClient())
            {
                T foundItem = ctx.GetById<T>(id);
                return foundItem; 
            }
        }

        public IList<T> FindByIds<T>(long[] ids)
        {
            using (var ctx = m.GetClient())
            {
                IList<T> foundItems = ctx.GetByIds<T>(ids);
                return foundItems;
            }
        }

        public void Create<T>(T entity) where T : class
        {
            List<object> list;
            if (!_cache.TryGetValue(typeof(T), out list))
            {
                list = new List<object>(); 
            }
            list.Add(entity);
            _cache[typeof(T)] = list;
            Store<T>(entity); 
        }

        /// <summary>
        /// Delete single {T} from cache and Data Store.
        /// </summary>
        /// <typeparam name="T">class</typeparam>
        /// <param name="entity">class object</param>
        public void Delete<T>(T entity) where T : class
        {
            List<object> list;
            if (_cache.TryGetValue(typeof(T), out list))
            {
                list.Remove(entity);
                _cache[typeof(T)] = list;

                RedisDelete<T>(entity);
            } 
           
        }


        public long Next<T>() where T : class
        {
            long id = 1;

            using (var ctx = m.GetClient())
            {
                try
                {
                    id = ctx.As<T>().GetNextSequence(); 
                }
                catch (Exception ex)
                {
                    Debug.WriteLine(ex.Message); 
                }
            }
            return id; 
        }

        public IList<T> GetAll<T>() where T : class
        {
            using (var ctx = m.GetClient())
            {
                try
                {
                    return ctx.As<T>().GetAll();
                }
                catch (Exception err)
                {
                    Debug.WriteLine(err.Message);
                    return new List<T>();
                }
            }
        }

        public void Update<T>(Func<T, bool> predicate, T entity) where T : class
        {
            List<object> list;

            if (_cache.TryGetValue(typeof(T), out list))
            {
                var existing = list.Cast<T>().FirstOrDefault(predicate);
                if (existing != null)
                    list.Remove(existing);
                list.Add(entity);
                _cache[typeof(T)] = list;
                Store<T>(entity); 
            }
        }

        public bool ExpireAt(string keyName, int expireInSeconds)
        {
            using (var client = new RedisNativeClient(ConfigurationManager.AppSettings["RedisServer"]))
            {
                return client.Expire(keyName, expireInSeconds); 
            }
        }

        public long GetTtl(string keyName)
        {
            using (var client = new RedisNativeClient(ConfigurationManager.AppSettings["RedisServer"]))
            {
                return client.Ttl(keyName);
            }
        }

        public void Set(string keyName, string content)
        {
            using (var client = new RedisNativeClient(ConfigurationManager.AppSettings["RedisServer"]))
            {
                client.Set(keyName, Encoding.UTF8.GetBytes(content));
            }
        }

        public string Get(string keyName)
        {
            using (var client = new RedisNativeClient(ConfigurationManager.AppSettings["RedisServer"]))
            {
                return Encoding.UTF8.GetString(client.Get(keyName));
            }
        }

        public IDictionary<string, string> GetInfo()
        {
            using (var client = new RedisNativeClient(ConfigurationManager.AppSettings["RedisServer"]))
            {
                return client.Info;
            }
        }

        public bool Ping()
        {
            using (var client = new RedisNativeClient(ConfigurationManager.AppSettings["RedisServer"]))
            {
                return client.Ping();
            }
        }

        #region Private methods 

        private void Store<T>(T entity) where T : class
        {
            using (var ctx = m.GetClient())
            {
                ctx.Store<T>(entity);                   
            }
        }

        private void RedisDelete<T>(T entity) where T : class
        {
            using (var ctx = m.GetClient())
            {
                ctx.As<T>().Delete(entity);
            }
        }

        private T Find<T>(long id) where T : class
        {
            using (var ctx = m.GetClient())
            {
                return ctx.As<T>().GetById(id); 
            }
        }

        #endregion 


    }
}


Sample console application using the RedisMemoryProvider:

using System;
using System.Linq;

namespace RedisProvider
{
    class Program
    {
        static void Main(string[] args)
        {

            RedisMemoryProvider<User> r = new RedisMemoryProvider<User>(); 

            // We do not touch sequence, by running example we can see that sequence will give Users new unique Id.

            // Empty data store.
            Console.WriteLine("Our User Data store should be empty.");
            Console.WriteLine("Users In \"Database\" : {0}\n", r.GetAll<User>().Count);

            // Add imaginary users.
            Console.WriteLine("Adding 30 imaginairy users.");
            for (int i = 0; i < 30; i++)
                r.Create<User>(new User { Id = r.Next<User>(), Name = "Joachim Nordvik" });

            // We should have 30 users in data store.
            Console.WriteLine("Users In \"Database\" : {0}\n", r.GetAll<User>().Count);

            // Lets print 10 users from data store.
            Console.WriteLine("Order by Id, Take (10) and print users.");
            foreach (var u in r.GetAll<User>().OrderBy(z => z.Id).Take(10))
            {
                Console.WriteLine("ID:{0}, Name: {1}", u.Id, u.Name);

                // Lets update an entity.
                u.Name = "My new Name";
                r.Update<User>(x => x.Id == u.Id, u);
            }

            // Lets print 20 users from data store, we already edited 10 users.
            Console.WriteLine("\nOrder by Id, Take (20) and print users, we previously edited the users that we printed lets see if it worked.");
            foreach (var u in r.GetAll<User>().OrderBy(z => z.Id).Take(20))
            {
                Console.WriteLine("ID:{0}, Name: {1}", u.Id, u.Name);
            }

            // Clean up data store.
            Console.WriteLine("\nCleaning up Data Store.\n");
            foreach (var u in r.GetAll<User>())
                r.Delete<User>(u);

            // Confirm that we no longer have any users.
            Console.WriteLine("Confirm that we no longer have User entities in Data Store.");
            Console.WriteLine("Users In \"Database\" : {0}\n\n", r.GetAll<User>().Count);

            //Do some misc additional test 
            r.Set("Dog", "Gomle");
            string dog = r.Get("Dog");
            Console.WriteLine("Key: Dog, Value: " + dog);
            r.ExpireAt("Dog", 11);
            long ttlDog = r.GetTtl("Dog");
            Console.WriteLine("Key: Dog, Expiration: " + ttlDog);

            var info = r.GetInfo();

            Console.WriteLine("INFO:"); 
            foreach (var x in info)
            {
                Console.WriteLine(x.Key + ": " + x.Value); 
            }

            Console.WriteLine("Hit return to exit!");
            Console.Read();

        }


        public class User
        {
            public long Id { get; set; }
            public string Name { get; set; }
        }


    }
}


So by using the code above, we can get started with using Redis.io in our C# based solutions much easier. The provider works on types, so your business entities will be divided into Sets in Redis.io according to their given type. Hopefully, this will cover many different uses. In addition, a local ConcurrentCache is kept to get quicker execution. Make note if you use this Redis provider in multi-tier environments such as load balanced clusters, you would want to refresh the cache now and then. The syncing of the content is of course not being kept if there are several writers to the cache. In that case, we might want to pump out events to reload the cache. Redis.io support both publish and subscribe, such that informing your consumers that the Redis cache is updated is a possibility. Redis is primarily being used for performance enhancement, but getting the cache to remain synced with a local ConcurrentCache above accross multiple tiers (nodes) will be a challenge.

9 comments:

  1. Hi Admin,
    Excellent blog and its totally loaded with valid posts on Java and .Net technology. Consider including RSS feed in your blog, so aspirants like me can follow your blog easily. .Net Training in Chennai

    ReplyDelete
  2. Thank you for the kind comment murali! I will look into using an Atom RSS feed or similar to provide a newsstream here.

    ReplyDelete
  3. Install-Package ServiceStack.Redis.Signed can be used.

    ReplyDelete
  4. Thank you for the kind comment murali! I will look into using an Atom RSS feed or similar to provide a newsstream here.
    base-sas training in chennai

    ReplyDelete
  5. In case you are looking into making money from your visitors by popunder ads - you should embed one of the most reputable networks - Clickadu.

    ReplyDelete
  6. This is excellent information. It is amazing and wonderful to visit your site.Thanks for sharing this information,this is useful to me...
    Android Training in Chennai
    Ios Training in Chennai

    ReplyDelete
  7. I recommended your blog post very helpful to us.The amount of information in here is stunning like you practically wrote the book on the subject.
    Selenium Training in Chennai
    Selenium Automation Testing Training in Chennai

    ReplyDelete