Having a large amount of data, no matter what format it may be – traditional paper files, microfilm or index cards, or a whole host of electronic formats used – is a good idea, Metadata Structure, why do it? And indeed, we live in an information society where everything we do depends in one way or another on a person’s ability to access large amounts of data very quickly and accurately.
However, the usefulness of that data – ours, utility companies, Amazon’s, state departments’, motor vehicles, or your health care provider’s, really any entity we do business with – is entirely dependent on being able to find the data when needed. That means you need a way to organize everything so you can find that data. And the larger the data set, the more this rule applies with the help of an M&A advisor.
You can scroll through a stack of a few dozen sheets of paper to find what you need without much difficulty. Now imagine a hundred. It is much more difficult. A few thousand and lo and behold it’s even harder and longer – and mistakes can creep in. A million – no chance. And the reality is that a commercial records system, however small, will be much more like that million – or even more – pages and not a few thousand, whether it’s electronic or on paper or whether it acts as a mixture of the two.
The consequences are simple but important: you need a method of organization, otherwise, you will never find anything – most of your information will be lost. And if you have an organizational method, the usefulness of your information corresponds directly to the quality of your method. Indexes, metadata, and data structures are all about bringing that order to your data set.
Is your metadata in the form of a panel or a card?
The problem could be considered by analogy. If you have ever driven a car in a very big city, London, or Paris for example, you know that it is very difficult, to say the least. And so very often those of us who haven’t memorized the whole map (as London taxi drivers do) can only attempt to navigate using a map or suite to detail instructions.
Without street names, this would be downright impossible. But since this metadata is in no way organized, its value is limited. That’s why taxi drivers memorize everything.
Compare that to my city, Denver. The whole city – basically the whole metropolitan area – is laid out on a grid. This organizational fact alone simplifies navigation. Colfax Avenue, for example, runs in a straight line from east to west for 65 km. But the grid is also numbered. There is a street corner which is the zero point – zero east-west, zero north-south. And in addition to a name, each street has a number – 100, 200, etc.
So, if I tell you that Pennsylvania Avenue is 500 east, you know exactly where it is. And if I tell you my office is at 4340 South Pennsylvania, you know that’s 5 blocks east, 43 blocks south. And they went even further, naming city blocks after consecutively numbered avenues after trees, historical figures, and the like. The side streets in my neighborhood are all named after colleges – I’m a few blocks from Pennsylvania and Oxford. Once you know the naming system and conventions, Denver is a very easy city to get around.
Dataset management is not too far from this type of comparison. Without information about what’s in the dataset, you must rummage around like you’re driving around London without a map and any signposts. So, you need metadata. And the more order and uniformity this metadata presents, the more effective it will be.
Filing systems – metadata for paper files
A well-designed and well-managed filing system is a good example of this. There is a range of commonly used filing systems – numeric, alphanumeric, terminal numeric, etc., but they all achieve the same important goals:
- Files have systematic and predictable metadata tags attached to them
- Files are stored in systematic and predictable locations
- Files on the same or similar topics are either grouped physically (by location) or grouped logically (according to code on file labels)
Applying these lessons to metadata for electronic systems
The same logic also applies to electronic systems. When you look at something like file structure in Windows Explorer or Mac OS Finder. You are looking at the direct electronic analogy of this paper filing system. Fashionable for a period. The Internet Central advocated that you didn’t need a systematic data structure. That freeform metadata tags were the only tool needed. But that fashion has stopped quickly because free-form metadata quickly becomes inadequate. As the number of data objects in the collection grows. If you consider this in the context of a paper-based filing system. You quickly realize why: If you have, say, a thousand filing cabinets to fill. Filling them all in random order with folders of unlabeled files would be a system completely useless.
Randomly labeling the folders with whatever comes to mind at the time would be a bit better… but not that much. It would still be very difficult for you to find a particular folder. It’s only when you consistently label folders and consistently categorize them that the setup starts to work. The same is true for electronic systems. Data objects are not necessarily physically contiguous, but the metadata schema logically associates them to achieve the same result through M&A advisory.
You often hear someone say, “We don’t use an index, we have a metadata schema” Is metadata different from an index?
No, they are not: an index is a type of metadata schema. Consider this simple index:
- Accounting
- Accounts Payable
- Accounts Receivable
- Human resources
- Applications and resumes
- Personal files
Each data object in this system will have at least two metadata tags associated with it, for example, “human resources” and “Metadata Structure”. The first places the object in a particular group of data objects and the second place it in a smaller subgroup. Each personal folder will then have at least one additional metadata tag in the form of a name or employee ID to allow the identification of a particular file.
When we consider such a system as a folder structure, it is important to remember that the folder structure does not exist. The objects stored there are usually stored on the hard disk – often several hard disks – at random. It is simply a graphical representation of a structured, hierarchical metadata schema. When we drop a file into a folder, we just attached a metadata tag to it.
Powering a Better Search Experience with Deep Metadata
The benefit of a well-designed automated system is this obvious. Author, keywords, the list is virtually limitless. Giving you allows for efficient searching – “Show me all payable invoices created. By Joe Smith between January 10 and July 7 relating to Acme Corp.
But to put this power into practice, your plan must be systematic and orderly Metadata Structure. It’s about predictability and uniformity of power. If we don’t always label them as invoices or always put them in Joe’s name, the system won’t work.
Your goal in creating the index is to provide the shortest possible search path to anyone searching the system. You can’t optimize the index for every type of search, so you optimize for the most frequent ones.
So, if you’re primarily looking for all tax forms for a single year, the second system makes more sense. If you mainly deal with one type of form over several years, the former makes more sense.
Success is all about planning
So, in building the index and other metadata tags, you first need to have an idea of who is searching. How that person does their job. And how much they want to search for things Metadata Structure. The same goes for the index terms themselves – they need to be meaningful to your users or they will be searching randomly. It’s also worth noting that in a good electronic system. You might be able to rearrange your index and display it in other ways too. Better ordered, combined with other metadata fields, horizontal, moved, etc.
So, in our accounting example, both layouts could be available as needed. The ability to make these dynamic changes on the screen allows for powerful searches that would not otherwise be possible. But always, awareness and care in crafting the terms and structure of your metadata. And a high degree of consistency in its application are essential.
You don’t need a powerful engine to have an efficient metadata index and schema. Also, the capability of a powerful engine without a good metadata schema will never be Metadata Structure great. The challenge is always to build the intellectual capital – index, data structure. Metadata sets, or whatever you call it – that will drive the system.