Search, Metadata, and Bye Bye Folders

Cross posted from AIIM …

Inspired by a recent project. Explained to my (almost) 12yr old. Guess who “got it” quicker.

Looking Back

Unless something happens that requires a pretty quick response, this will be my last post of 2011. And since it’s the end of the year I thought I’d do like so many others and take a look back. However, I’m not looking back at stuff that happened during the year; I’m going back to 1991, to a project and period in my career that I really, really enjoyed. I consider it the first ECM project that I worked on, though I’d never heard of ECM at the time. It’s also the first independent consulting gig that I took on.

In September of 1991 my wife was asked to move from the Montreal area to the Toronto area for what was supposed to be a maternity leave replacement as one of her colleagues had just had a baby. Since the alternative was unemployment for her, and since I wasn’t working anyways (I was back in school) we decided to make the move. The company she worked for was a small distributor of electronic components (switches, plugs, receptacles, etc.). Their clients were mostly in Canada, with a few in the U.S. Their suppliers were predominantly in Europe and Asia, with a few in the U.S. They also had to deal with transport companies and customs brokers/agents in order to carry on their core business.

My wife’s job was to handle sales orders, purchase orders, customer service, transportation, and customs (I did say it was a small company). Other roles in the organization included one accountant/bookkeeper, some sales guys, a general manager, a warehouse dude, and the owner (one of the nicest guys I have ever worked for or with). I was engaged to implement some accounting / order processing software for them (ended up selecting ACCPAC vWhateverWasCurrentIn1991). I eventually ended up doing double-duty as the IT / Warehouse dude (regular whse dude ended up in jail, I ended up reporting to my wife – oh the fringe benefits).

This company was really excited to be getting their first computerized system of anything. Included in the bundle of stuff that they were getting was a rockin’ 24-pin, tractor fed, colour, dot matrix printer to handle their multi-part forms. They could only afford one colour monitor so there was a discussion about who’d get it and who’d get stuck with the grey-scale monitor. The fax machine used rolls of thermal paper (I’m sure at least one of you reading this has no idea what I am referring to). Printing to the correct printer was controlled by turning the switch on the printer switch box (I still have one in my basement somewhere). Everything, even the stuff that was born digital, was stored in filing cabinets, and everyone had their own copy (we hoped), whether or not they needed it.

Every customer, supplier, and vendor had a file. These files were kept in the GM’s locked office, which wasn’t generally a problem until his love life fell apart and he started taking “mental health days” several times a week. Orders from the customers came in by phone or fax. Faxes were great, especially when several orders came in overnight and needed to be separated from each other using scissors. Confirmations from suppliers were received by phone, fax (see point about customer orders), and mail (sometimes after the order itself). Customs paperwork came in by courier (time was of the essence) or fax. Outbound paperwork was couriered, faxed, or mailed. Sometimes the same document was delivered by all three methods, just to be sure. Internal – internal paperwork was walked to the recipient, except for the out-of-town sales guys who got their stuff via fax (including a nude picture from a calendar we got from a Taiwanese supplier – very tasteful, too bad I sent it to a customer by mistake, oops).

But, they had ACCPAC now. I don’t remember exactly which modules I implemented for them, but G/L, A/R, A/P were part of the mix. Not only did they have automated accounting software, we even went so far as to try to eliminate some paper duplication and streamline some processes. There was certainly some resistance (mostly from the accountant/bookkeeper but she went batty, got fired, and hid a CDN$36k tax bill in the customer invoice files). The point is that there were improvements made, even if we couldn’t apply automation to everything. And I made really good money (by the standards of the day), wasn’t on the clock 24/7 (no cell phones or internet), had plenty of time for me (and my wife/boss), and delivered a project I felt really good about. As things turned out, the company offered my wife and me permanent positions, but we missed Montreal so declined and headed home.

If I had to do that project over again today I would, assuming that the client could afford my rates. I’ve learned a lot (and made mistakes from which I’ve learned) over the last twenty years, so I certainly wouldn’t do things the same way, process or approach wise. The biggest difference, however, would be in the technology. Not only are the tools we have available today much better than what they were back then, they’re enormously less expensive when you consider what you get for your money. In fact, I’d guess that much of the software they’d need for today’s conditions could be had from truly viable open source sources. So maybe they could afford me, after all.

I was going to write a long-ass section detailing what I’d implement to handle which bits of the business, but I changed my mind. We all have an idea of what we’d do with this type of a project.

Records Matter, Declaration Doesn’t – Revisited

This is a response to comments that Jürg Meier made recently on something I posted a while ago. Jürg is a very smart and personable guy, whom I had the pleasure of meeting in person at ARMA Switzerland’s inaugural event on November 29, 2011. I urge you to check him out on twitter and here, where he works.

I was going to simply reply to Jürg’s comments on my blog, but I figured that the points he brought up are pretty substantial and would be of interest to a broader audience. I asked Jürg if I could paste his comments into a post and respond to them. You’re reading this so either: a) Jürg agreed; or 2) I’m in deep doo-doo.

From the original post: “Users know what business process they’re involved in. …”

JM: Chris, not sure here. What about knowledge workers, who “often advance the overall understanding of that subject through focused analysis, design and/or development” (Wikipedia). Are they in a business process? Perhaps, but more often than not in a very large one, like a product development, an IT or marketing project. These people send email, word and powerpoint docs back and forward, take notes. Notes? Karl Alexander Mueller, Nobel price winner in physics 1987, discovered a material for high-temperature supraconductors. He took the decisive note a few years earlier at a congress – on a single page of his pocket notepad.

Moreq2010 comes up with a similar example. In the introduction chapter, they show a hand written shopping list and the resulting cash register receipt. They consider only the latter as a record.

I would say that regardless of what the duration or intended outcome of a process is, it’s still a business process with measurable business objectives. Projects and cases (as in case management) cross multiple business processes and can be of several years duration. Product development takes ages, is complex, involves large numbers of people and huge volumes of content. However, it can still be tied to business processes and the participants (usually) know what they’re doing. In that type of scenario I think I would recommend using a case aggregation for the users to plunk their content into, and apply appropriate retention to the aggregate.

I would assume that Müller knew what he was working towards when he wrote the decisive note in his pocket notepad (how different is the pocket notepad from a tablet these days?). If my assumption is correct then it stands to reason that the note is part of the research documentation, which must be filed and retained. The big question in my mind is related to ownership of the intellectual property; does it belong to Müller, to IBM, or to both?

Another question that I have concerns an outcome that is unintended, but beneficial nonetheless. I’m sure we all remember what Sildenafil was originally intended for, and what its current use is. What, if anything, are the impacts on categorization and retention? Research & knowledge based processes are really tricky to deal with, but I think the key is that you can apply (business) rules & automation to the mundane aspects, use aggregations to capture the content, and let the participants do what they are engaged to do. I would certainly rather have medical / pharma researchers figuring out cures than worrying about where to file something.

The Moreq2010 shopping list / receipt example is analogous to an order / invoice example. Each document provides a part of the complete picture, and therefore is required. I also think that particular example is nonsensical unless for some official reason (e.g.: personal taxes) you need to hold on to the receipt. Frankly, I need to keep the list to prove to my wife that I didn’t bugger something up when she “let” me go shopping for her.

JM: In my experience, it is really a question of who will consume the information. There are the usual suspects:
- business
- legal
- long-term (historical) archive

As you pointed out during your speech at the Swiss ARMA Chapter inaugural meeting, different people have different views on the same information. So, it would be compelling to classify information multiple times by different consumers… and I’m inclined to say: as late as possible. Only if we know about the purpose of the classification, we can do it right. E.g. for legal, they actually only know what they are looking for upon a litigation. By then though, they know very well what they need.

But what’s wrong with classifying as soon as possible, and adding additional classifications as they are identified, if that’s the case. The classification with the longest retention drives how long any content needs to be kept. This only works when classification and retention/disposition are segregated. In litigation situations simply applying a hold / freeze will do the job. There’s no reason to apply additional classification to the content because you create a legal case file aggregation and dump the content into it.

Content that has archival value, but no risk is easy – just keep it. I mean, I know we’ll want to keep all of my blog posts for the next 300 years or so. J It’s tougher when content that has archival value has some potential risk associated to it (privacy issues, legal exposure). I think at that point it’s really a judgement call. Frankly, I’m in favour of preserving because I’d like to think that sometime in the future there are going to be people that are interested in what we’ve been thinking and doing, and that the information they want is available. I’m also hoping that we’re not so stupid that we evaluate everything in terms of whether or not we’re going to get sued.

JM: However, the case of “late classification” does not answer one key question: for how long should we retain? The only reliable basis here is law and the retention schedule. And for that, by nature, we must classify upfront. That isn’t too difficult for “real business processes” (e.g. selling a ticket), but becomes tricky with output from knowledge workers. Here, to some extent, we need their support. Classifying draft/final is a good start, formally assigning it to a project would be very helpful, as well as identifying ownership and the document type.

For the most part I agree with this paragraph. For the knowledge workers, especially those that are involved in a lot of trial and error, I think we can come up with some reasonable classifications and retentions for them to use. Imagine how different things would be if the people that were working on Sildenafil tossed everything away once they realized they weren’t going to achieve what they set out to do.

Cloudy with a Chance of Success

This was originally posted on the AIIM Community on November 18, 2011.

This post was inspired by this article on CMSWire by @billycripe and by the Cloud themed tweet jam hosted by CMSWire on November 17, 2011. As usual this is just my opinion.

I’m not an expert on cloud computing, I’m just some guy that likes to be able to access the content I need to do my work, from wherever I happen to be, using whatever device I feel like using at the moment. Take this post, for example; it was written on a laptop and a tablet, in a dining room and a swimming pool (not really in the pool since my tablet isn’t waterproof though that would be mega-cool).

I agree with Billy Cripe’s thoughts that Agile can (ought to) be applied in the development of cloud based ECM solutions. However, as Billy correctly states, “Managing content is not the goal of most businesses.” Most businesses exist to make money by providing products and/or services that consumers want. Businesses rely on information in order to get their stuff done, whatever their stuff is. In order to fully exploit information, the tools (i.e.: information stores) that the businesses rely on need to be connected to each other (so do the people – the tools need to facilitate this). Content / information management tools (cloud or not) need to be part of bigger picture business solutions. We need to build solutions that deliver “I need to share this” in the context of why it needs to be shared (answer why you need to share and you’ll likely figure out who and what).

No sane person can argue the value and validity of the cloud. Except me. I’m not daft enough to think that cloud computing doesn’t have value or is not a valid approach to take. However, I do think that we’re not going to realize the full potential of the cloud (and by extension, content) if we simply limit its scope to content management. Yeah, I know that there are other things that are done in the cloud, such as CRM, payroll, and accounting.

Content Wherever I Am

One of the cool things about content in the cloud is that my content is wherever I am. (Okay, so it’s not really my content, it’s my organization’s content.) That’s not the point, though. The point is that I can work with content wherever I happen to be, using whatever device I choose. This does assume that the chosen content repository is able to be synched appropriately. Wouldn’t it be cool, though, that if in addition to being able to work with the content and share it with collaborators (the work variety, not the WWII Nazi variety) the content could also be appropriately tagged, filed, and placed under retention at the point that I plunk it into the repository? I.e.: Cloud repositories need to become extensions of ECM and ERM systems, probably through federation.

Correctly Connecting Corporate Content

Content is spread throughout an organization; cloudification just increases the spread. When I say content, I mean anything that is stored on digital media that serves any legitimate business activity. (For obvious reasons I am excluding physical content.) A key to widespread cloud acceptance is to be able access / leverage content in order to execute a business activity, regardless of where the various pieces of content reside. An agent in a social services organization should not have to know or care that a citizen’s information is spread over a number of repositories that could be on-premises, in a private cloud, and in a public cloud. The agent is there to service the needs of the citizen, not to figure out some (likely) convoluted architecture just to try and find stuff.

CMIS is a step in the right direction, but where CMIS falls short is that it doesn’t address non-CMS (think ECM) repositories. What we need is something that allows connecting everything that we need, when we need it. Device and location should not be factors. In fact, the only thing that a user should worry about is whether or not they have the right content to do the job. Governance, classification, and security ought to be just taken care of.

Speaking of Governance…

Until the governance issues get sorted, I doubt very much that we’ll see widespread adoption of public cloud services. Smaller organizations, organizations with lax regulatory / privacy regulations, and organizations that can bully providers into rock-solid SLA’s may be able to go full public cloud, but I doubt they will. I think the reality is that organizations will end up having hybrid environments of cloud and on-premises.

When I say governance I am not only referring to the poo that legislators, regulators and litigators throw in our way. Governance needs to address issues such as:

1. what can / should be stored in the cloud
2. service level agreements
3. disaster recovery / business continuity
4. security
5. classification / categorization
6. retention & disposition (thanks to @JamesLappin & @AlanPelzSharpe for bringing this up)

Governance of cloud content has to deal with all of the things that we need to deal with for on-premises stored content, with the added complication that we also have to deal with where the damn box is and if some foreign government can get at it whenever they bloody well feel like it. Canada’s Anti-terrorism Act and the United States’ PATRIOT Act are not going to be very helpful in encouraging organizations to move to the cloud in a big way.

Parting Shots

1. Hybrid (cloud / on-premises) will be in the majority
2. Governance (internally & externally imposed) has to be figured out
3. Integration / interoperability are critical
4. Privacy concerns and government snooping are major inhibitors (@ron_miller wrote a pretty good piece about this)
5. If we’re not careful we’ll just move the mess from our hard drives to someone else’s
6. Some Systems of Record will end up in the cloud, if they’re not already there
7. Services are where it’s at

Bonus Material

I couldn’t decide which song I wanted to use for this post, so you’re getting three:

1. CCR – Have You Ever Seen the Rain?
2. CCR – Who’ll Stop the Rain?
3. SRV – Couldn’t Stand the Weather

A couple definitions for those that think it should be “on-premise”

1. http://oxforddictionaries.com/definition/premise
2. http://oxforddictionaries.com/definition/premises

Come Together

Like Adam said to Eve; “Stand back, I’m not sure how big this thing’s gonna get.”

Over the last few weeks I’ve been thinking/ranting/blogging/tweeting about unstructured content, social business, systems of record (SOR), Systems of Engagement (SOE), mobile stuff, and the whole thing about going paperless. What I’ve concluded is that it’s all seemingly one big inextricable mess. It really isn’t, though. I think the key pieces are Systems of Record/Engagement (SOR/E) and interoperability (CMIS, Integration for the smarty-pantses reading this).

Regardless of what some want / think, we’re not going to ever be 100% paperless even though it makes lots of business sense. The uproar over the mobile thing is a temporary distraction caused by all the new devices we have at our disposal to create “stuff” and interact with it.

SOR/E’s and interoperability are where it’s at. However, you first need to understand that a system of anything is not a tool. D’uh. A system is the combination of the processes, people, information, and tools that are used to get sh*t done!

A system that includes laptops, tablets, smart phones, and mainframes is every bit as valid as a system that includes pens, paper, and abacuses (abaci?). The key is to combine the various system components in a way that achieves an intended outcome. Unintended outcomes of a positive nature (serendipity, opportunity) are welcome. Unintended outcomes of a malevolent nature (issues, but you better have a risk management plan and understand the difference between issues and risks) are not.

Mobile devices (Ha! My brain is mobile and goes where I go. Does yours?) and mobile content are today’s red-headed step-children. But that’s only because they’re relatively new concepts (aside from cell phones and txt msgs) and we haven’t really figured out how to manage the content they produce, consume, and interact with (Actually, that’s crap; devices don’t do content, people do).

In his AIIM white paper, Geoffrey Moore gets one thing kind of right: “Clearly, systems of engagement need to operate on top of and in touch with our existing core systems of record.” I’d state it a bit differently; SOE’s extend the possibilities and reach of SOR’s. SOE’s and SOR’s need to be merged in order to achieve true Enterprise value.” (That’s why I came up with SOR/E’s).

I think he (Geoffrey Moore) also gets one thing dangerously wrong, though I don’t actually think it’s intentional. It may just be me, but I get the prickly feeling that when he refers to system he actually means tool (e.g.: PeopleSoft, Siebel, ACT!).

What if we were to take a different look at things? What if, instead of systems of records and/or engagement (you’ll notice the intentional lack of capitalization) we looked at them as layers; as in an n-tiered architecture? Is the engagement layer really anything more than what the user sees/uses? Isn’t the record layer merely the database / content repository? Are devices really anything more than elements of the presentation (think User Interface) layer (a.k.a. the Engagement layer)?

By the way, I am not using the word “records” in a Records Management (ARMA, CRM) way here. Records really ought to be thought of as transactions.

Why did I use the Aerosmith version instead of the Beatles original? Watch it and figure it out.

I’m Goin’ Back

I left Montreal in 1997 because of the political, social, and economic climate in Quebec at the time. However, even after all these years I still consider Montreal my home. The Quebec government wanted to protect Quebec’s language and culture, which I don’t object to. What I do object to is how they went about it; at the expense of other cultures and languages, especially English. They called themselves a “distinct society”; I viewed Quebec as an insular or isolated society. You see, they decided that they were so special that they didn’t really need to play nice with anyone else. They were wrong.

Within Canada, Quebec is certainly a distinct society and, to quote Martha Stewart, “It’s a good thing.” Times and the world have changed. Quebec can longer isolate itself from the rest of the world. I think this realization has even made its way into the consciousness of many Quebecois who previously had the opposite view. Why do I think this?

Since the middle of June I’ve been commuting to Montreal for a project. I’ve had many conversations about the state of affairs with many “experts” including my Dad, bartenders, cab drivers, tourists, colleagues, and likely a few random lunatics. There’ve been common themes in most, if not all, the conversations that I’ve had:

1. Quebec’s economy sucks;
2. Quebec realizes that there’s a great big world out there;
3. Quebec needs to participate in the global economy;
4. I probably spend too much time in drinking establishments talking to strangers.

Now here’s the Information Management bit…

We’re all well-aware of how social media played key roles in the Middle East over the last several months. We all recognize the power of social media to facilitate action during natural disasters. These types of events are fast paced and immediate (when measured against our existence). But, can social media, and online existence in general, be a factor in change that takes a longer time to come? I have a little theory that were it not for the internet and all the goodness it purveys, Quebec would have taken a lot longer to come to where it is today in terms of thinking about its place in the world.

Quebec still has a long way to go, but once they get there (and my parents relocate to maintain a 2,000 mile buffer)… I’m Goin’ Back Again (apologies for the crappy quality but it was all I had the patience to find).

Mythical Beasties

It seems I’m not the only one that doesn’t believe in the myth of unstructured content. Like Sasquatch, Santa Claus, and Ogopogo (CRTC mandated Canadian content), we’ve all heard about it, but have we really seen it? I mean really seen it? Not like some mirage or something. (You’ll notice I made no reference to peyote induced visions.)

The one type of unstructured content that comes most readily to mind is the document that contains text – uh, word processing documents. Even before we had word processors I’d still argue that text based documents had some structure; whether they were produced by a typewriter or written by hand makes no difference. Even that letter you write to Santa Claus every year has structure. Dates, salutations, body text, closing, … they are all structural elements of a document.

Remember those three-part memo forms the nice lady with the pointy brassiere used to fill out for the boss? The one that said “MEMORANDUM” at the top. Uhm, “MEMORANDUM” is a metadata value. And if metadata does not provide structure, what does?

 

“What about pictures, Uncle Chris? They don’t have structure.”

Yes, Julie, they do.

Digital photograph files are loaded with metadata, probably more than is really useful for most organizations or people. Even the old fashioned photos that had to be developed manually had metadata associated to them. The major difference was that the metadata was usually in the photographer’s or subject’s head. Think about those notes that many people wrote on the backs of their pictures. Think about the photography geeks talking about f-stops, exposure settings, zoom lens size, … Okay, I’m officially bored now (no offense to photography buffs intended).

Music, paintings, sculpture, even the human body; they all have structure (the lady next to me on the plane has really nice structure). My point is that everything can be described and categorized. Like much of what has changed, nothing has really changed except our ability/need to do things faster. For the most part we are doing the same things, just better. The big thing is that today we’re able to, relatively easily, capture metadata and store it in order to make better use of it. It has always been there, though.

Like most myths, the myth of unstructured content is likely born out of some factual occurrence, but distorted over time and telling. Whether the distortion is wilful or innocent is anyone’s guess.

Another smart person’s take on the subject.