I'm often asked how one gets the skills required to do my job as a Metadata Librarian. My answer is one I can't stress strongly enough: experience. We need to know the theoretical foundation of what we do inside and out, and need to constantly think about why we're doing something - the big picture. But theory is not enough. The only way to become skilled at making good metadata decisions is practice--seeing what happens as a result of an approach and improving on that approach the next time. No matter how many times I've done a certain type of task, I see the next repetition as a way to re-use good decisions and re-think others.
I've found the metadata community in libraries to be a very open one. When I'm starting on a task that I haven't done before, I use what I can from my experience with similar tasks. But I also ask around for advice from others who do have that experience. "Metadata" is a very big and diverse area of work. Even with the best abstract thinking, applying known principles to new environments, we all often need a boost for getting started from someone who has been through a given situation.
I'm skeptical of the idea of "experts" overall. These things are all relative - only once you start learning enough to be able to effectively share what you've learned with others do you truly realize how much you still have to learn. I put much more stock in the goal of becoming good at thinking about generalized solutions, good at making decisions for classes of problems rather than simply repeating specific implementations over and over. I'm not a programmer, and neither are many in the metadata librarian community. Yet this type of thinking that makes a good programmer can, in my opinion, make the best metadata experts as well.