missmiis

ILM,Virtual Directory,Abstraction Layer

Miss Miis (missmiis - clever huh?) made some comments I wanted to bring attention to in a recent blog posting.  She keeps up on ILM/MIIS happenings and her blog postings are useful to people using these tools.  Recently she was at the TEC conference in Las Vegas and got a chance to learn more about Virtual Directories from Todd Clayton.  She makes a clear and easy to understand explanation of virtual directories; to deliver identity data in the way applications want to see it.  This abstracts the complexity of data silos from applications, making it easier to manage your data, without disrupting your environment.

She properly compares the ILM approach to using Virtual Directory as to synch data around vs. keeping data where it is and presenting the distributed data as a single LDAP source.  She seemed very excited and wanting to learn more about this technology.  The use cases for virtual directory are many, and I believe they will continue to expand as people become more aware of the capabilities.

Check out her post and take a look around her blog at the same time.

why cache and virtual directories???

I know I have mentioned this before, but since there’s an ongoing ping-pong match about cache — particularly “persistent cache” — playing out on the identity blogs lately, I thought I’d return the volley. Ashraf Motiwala covered the topic again in his blog yesterday, so here are my two cents.

First off, while it’s true there are times when cache makes no sense, there are other times when it really does.  Cache is used everywhere in your PC, software, servers, EVERYWHERE —so arguing against cache seems completely strange to me.

Second, I always find it funny to hear the arguments against having more options. Why argue against choice and options? I can cite many projects that have been deployed with caching using virtual directories, and yes, this includes "persistent" cache. 

Third, cache is necessary because when merge multiple tables (join) across different databases (or directories for that matter), the results are just not fast enough for any type of security application. Anyone familiar with databases should understand this quickly. Once you join several objects or tables, the response rate of the source is dramatically reduced. The joins necessary to create views are sometimes too complex to do on the fly for most directory-enabled applications, such as would be common for IdM/security. This, in my mind, is a key functionality of virtual directories after aggregating sources for a common protocol.

Fourth, 2 to 5 milliseconds can be a big deal, and cache is essential to eliminating that lag. Think about it, if I have to search for a member in a directory, and then search a database table for additional attributes to join to this object — do you really think it will perform at close to the same speed? And that is with just two sources...imagine the performance hit you’d take by adding additional sources and multiple join operators.

When your directory is expected to perform at 8000 queries a second, adding 2 to 5 milliseconds can be a VERY big deal. OK, let's keep the math simple and take a closer look what the problem is…

• I have a directory that performs at 5000 q/sec
• That translates into .2 milliseconds per query (5000/1)

• Your "overhead" is 2 milliseconds (the best performance cited)
• My queries now take 2.2 milliseconds (11 times slower)
• Now instead of 5000 q/sec when I access my directory I get only 455 q/sec

Some people might argue this is not a "minimal" performance hit. There are many initiatives where this type of speed would be totally unacceptable. This is actually a perfect case where persistent cache would be helpful. A persistent cache could easily bring this query rate back up to 5000 q/sec (or higher), even in the case of more complex operations such as more than two sources and more than one join. 

Fifth, the idea that you compromise the "freshness" of data for the sake of speed misses the point about what sort of information we’re dealing with here. We’re dealing mostly with identity using directories (people and other objects), and the identities themselves do not change very often in comparison to other data, such as transactions where updates and write operations are more common than search/query. For example, in your bank account, your “identity” information (name, address, phone, pin number, passwords, etc) changes far less your balance and activity.

The idea is correct, cache will create a lag in updates being available to client applications, BUT virtual directory implementations using persistent cache with event-detection cache-refresh mechanisms, offer (near) real-time incremental updates of information. 

Furthermore, if an account is disabled, and it takes 1 second to update that account to all cache instances this would be an improvement. Currently many organizations currently are taking several minutes or even a full 24 hours between this type of updating. 

Well, those are my 5 cents worth of comments... I know I promised only 2, but who's counting pennies, just milliseconds, right? :)

Oh, btw, no one has mentioned the distributed remote persistent cache story, which I have seen implemented with virtual directories — now we’re talking about some serious advantages... If anyone is interested, I would participate in such a discussion... 

Identity Management 2.0 ?

Darren Calman posting something that has attracted some attention from Matt Pollicove and from Ping.  I think it is worth paying attention to. You will find some good historical and pragmatic information, but of particular interest to me was what was stated about identity virtualization.

Virtual directories are being touted as a IdM 2.0 by Matt Pollicove, as a "identity virtualization" service.  Virtual Directories have been around for almost a decade now (8-9 years).  Perhaps their value is being increasingly realized.  The need for directories is not going away soon, their secure hierarchical structures give context and the high-performance needed for such things as web portals, which could be servicing millions of users.

I believe that the need for this type of value will continue long term, even if we see a decline in the use of directories and LDAP  There persists a real need for applications to understand the semantic relationships that are not easily represented in more traditional, and more prevalent RDBS AND in security or externally facing applications (which user volume is typically higher) a real need for high-performance and availability that can not be easily achieved through other database systems.  Using a thin virtualization layer on top of other databases to provide the functionality of directory for security, but not the primary storage of data, is an interesting evolutional possibility in the identity space. Security, search, and querry could still be serviced through this layer, but insert, update, delete operations could be handled by an RDBS, maximizing effeciencies.

The virtual directory offers this "virtualization of identity", solution, where relational tables can be presented as hierarchical views, accessible as LDAP or other protocols as needed. To acheive this value your "identity virtualization" service must offer:

1. data modeling (as to not constrain you to the existing structures if a different view is needed, but still maintain the existing relationships),
2. the ability to maintain high performance (because if back-end sources are NOT primarily LDAP, there are complex joins, or cross-application searches needed for authorization, performance will not be high enough),
   
3. a choice in deployment (proxy AND data model; dynamic AND event-driven update of an instantiated model (materialized hierarchical views) and
4. choice of protocols (not only LDAP, but a object oriented system that is more agnostic about delivery, at minimum you should get LDAP, SQL, and web services)

Perhaps we are about to pass the "hype" stage of identity management, but that means it is now is time for the heavy lifting to begin, the work of deployment and implementation.  Is your infrastructure ready?

 

Lost in the Jungle?

To add more thoughts and explanation to my last posting, "Are we lost?"

The idea is to not cut through the jungle every time we have a new initiative  At least solve the integration problem, once.  Don't reinvent the wheel every time you have a new application or new initiative.  Don't replicate data everywhere, employ complex synchronizations, and in general make everything 10X more complicated, compounded with each new deployment. 

Build one platform to plug everything into.  Sounds too good to be true?  Yeah, maybe it's not "everything" yet, but almost.  AND I believe this technology will evolve further and will become even more useful. 

This doesn't address the plethora of applications and methods available in the IdM market, but it does give you the opportunity to not care.  You can integrate any application you want, use any protocol, any schema, any data structure, any security means, any authorization/authentication schemes you want, as many as you want.  That is the power of virtualization. 

For me, within the virtualization platform you need proxy, data modeling, synchronization, service bus, and the ability to do build a correlation index (key mapping) if needed.  If you have these things within a virtualization "platform", then you will be ready to face the future.  You know where you can start, at least your identity stores are integrated into a common platform.

If you need LDAP, you have it. If you have the need for web services, use it. If you need to concatenate, transform, and otherwise alter objects, you have the tools.  If you need to query data, you can, if you need to push / synchronization data, you can.

The beauty is you do this work once; integrate your sources, then you just need to build a virtual object (e.g. in XML) to meet the needs of a new application or initiative and how you want to access it.  This is where you save time, money, and the headaches. 

You might still get lost in the IdM jungle, but at least you have a point of reference, a stronghold in the jungle.  A starting point instead of starting from scratch each time. This means you are free to choose a solution or application based on the business needs and its own merits, not your environment.  Otherwise you get lost in trying to calculate integration costs, deployment times, feasibility studies, and so on.

Solve what we can today, let others worry about the future. "Tomorrow has enough worries of its own." Simplify your workload, use virtualization to solve the problem of identity integration, don't stay lost in the jungle.

Are we lost?

There is a posting of interest recently by Pamela Dingle, you can check it out here.  I love to see people talk from their gut and talk about real issues, without the marketing guff and fluff, just real-life observations. Her recent posting "Catalyst Epiphany 2 - We're a little lost" points out the fact that identity management is all over the map.  I agree, sometimes I feel like companies are throwing solutions as fast as they can and hope that something sticks. Identity solutions have become a type of jungle, one that you never know exactly what to expect when you get inside.  Yes, there are explorers who have been there, and have reported back to us what strange and deadly creatures you will encounter on your own journey.  But where are the roads, the standards?

When my personal life gets like this I know it is time to slow down and take inventory, examine the issues. Perhaps this is where we need to start, to slow down and start to examine the underlying issues and what the real needs are in Identity and Access Management.  I have been to this crossroad, and I think this is why I believe so strongly in the identity virtualization platform (which virtual directory is a part of, and what started this concept).  It has given me an anchor point to start from; tools in my pocket like a swiss army knife to address the multitude of issues that plaque identity integration.

 

Single Sign-On

There was an SSO webinar today by Quest Software.  I would like to thank them for actually putting some content into it and actually explaining what their solutions offer.  I have become a bit weary of webinars so laden with marketing message, you really have no idea what the technology is offering you or if it will fit your infrastructure or not. 

The recording is available here.   My big take-away's had a familiar ring to them, as I have repeated some of these points too many times to count at this point.  Quest has an offering of multiple applications for an SSO package.  Also, check out Mr Shaw's white paper on the subject. 

First SSO means different things to different people; enterprise SSO, Federated SSO, Web SSO, Single Password, Reduced SSO, etc.  But I believe that some things are the same regardless of the "type" of SSO you want to deploy.

Shaw had a nice list of the "perfect world" for SSO:

• Standards based
• A single password or login
• A single directory
• Strong Authentication support / multi-factor authentication
• Support for multiple platforms
• Support for multiple applications
• Support for "thick, thin, and web applications"

This week I have been faced with some of the problems in solving SSO integration problems and how to deliver identity information to enable SSO.  A client had previously deployed a SSO solution for 35+ applications, using only two sources that were disjointed. Now they were faced with integrating another data source that contained intersection of identities. Some users in the new source exist in one of the other two sources - now the entire SSO solution collapses because it was based on the idea that they would never face an overlap of identities within the repositories.  Again, the answer was to deploy a virtual directory server as the abstraction layer to simplify the management of identities in these, now jointed data sources.

The lesson is, yes you can deploy an application without building a unified infrastructure, but you take a big risk.  When new data sources need to be incorporated (including acquired applications with their respective user-base), you can expect problems. 

PLAN AHEAD.. if you don't need to solve this integration problem now, YOU WILL at some point if you expect any level of scalability.  YES, it is possible to have one source for all your users, without replication.  By using the features of a virtual directory server to create a true union of identities (identities are correlated appropriately and duplicates are eliminated) and extend those entries with additional attributes from the needed data sources, you provide an identity infrastructure to provide authentication and authorization services to your SSO application. 

 

SOA in the IdM

Here is a definition of SOA given in the article found at http://www.itbusinessedge.com/item/?ci=23055 - its a short understandable definition;
"SOA. Service-oriented architecture refers to a paradigm that focuses on how you maximize the sharing, reuse and interoperability of distributed corporate resources across your network. And to maximize sharing, reuse, etc., you need a universal middleware environment, an integration fabric, a set of standards. So that comes down to things like the Web services standards" and I would add LDAP and SQL to these standards, don't keep this idea only in the world of external users, internet, or even intranet - use these concepts at the data integration level inside your IT deployments, especially in the IdM services space....

Virtual Directories are such a middleware component that can accomplish this. SOA is here if you want it, or you can wait until the vendors catch up and start helping you understand how to use their products....

Common Virtual Directory Scenarios

Excerpts from this posting... good stuff...

http://360tek.blogspot.com/2006_03_01_360tek_archive.html

"

1. Protocol Translation
   
2. Web Service Enablement
3. Multi-Repository Search
4. Joined Identity View
5. Permission-Based Results
   
6. Dynamic DIT
7. Authentication
   
8. Real-Time Data Access
   

Virtual Directory technologies eliminate boundaries. Hassles related to LDAP object types, attribute definitions and other schema-related issues are eliminated by virtualizing the view into the backend identity stores. You're no longer limited by the existing data format or database branding. There's no requirement to migrate the data from a relational database into an LDAP directory in order to make the data LDAP- or Web Service- accessible."

He talks about these issues quickly, but don't think they are not HUGE issues in deployment.

Also, some virtual directories offer more interfaces than standard just LDAP, he alludes to this fact as "web service- accessible", but still implies the use of LDAP for the web service. Some virtual directories could present information by other protocols such as DSML (for web services), SQL, SOAP, SAML, etc. Make sure the virtual directory you use supports different protocols for application access.

synchronization versus virtualization

virtual directories vs. meta-directory - most of the story is right on, but misses features of a virtual directory focusing on a virtual directory as only a proxy engine, which it is not. virtual directories can offer real-time synchronization AND persistent data, negating most of his "disadvantages". Meta is old, Virtual is new and more adaptive

read more | digg story