Scott Mark posted a piece a few months back about interviewing and interview questions. A common dilemma with interviewing as he reported is this:
I always struggle with what questions make great questions in this context. It's really too easy to focus on specific tools and frameworks - I don't think that's good. We try to keep these screens to a half hour, and the conversation can quickly de-generate into discussions around settings, API methods, and various extemporaneous details that make my mind wander or even want to explode - and before you know it time is up. I get very little qualitative feedback about someone hearing them go on about this kind of detail.
Since I'm a consultant, I have had quite a few unique experiences as an interviewee and interviewer, and I suggest that a resolution to these problems can be summed up in one sentence: What exactly are you hiring for?
Too many organizations have problems hiring mid-to-senior level technical people, in particular analysts and enterprise or software architects, because they don't have a handle on what exactly they are after. For example:
- I took a phone screen from a potential client for an 'architect' position a few months back. Had a 2-hour conference call with them that touched mostly on EA and business issues. While they also mentioned development and leading development teams, the focus was more on the business and requirements sides of the work. Near the end, I asked them what they felt the split was between development and business-related activities were. The answer came back "70% development, 30% architecture/analysis." This organization either had no clue what they really wanted or needed, or they thought that they could pluck Superman out of the woodwork who got the whole picture they envisioned right down to specific lines-of-code the first day. Completely unrealistic would be an understatement of this case. Most of the interview did not focus on hardcore technical topics, but then with the expectation that 70% of the time would be spent on them, they really didn't get the interview topics down well from their end.
- Another organization wanted an enterprise architect, but spent most of the one hour I had with them grilling me about technical topics and minutiae, which deconstructed into elaborate details similar to Scott's experience. That left me with about 3 minutes in that hour to get maybe 2 of the 10 questions I had about them answered and ascertain whether I really wanted to work with them.
- Other outfits call system architects and team leads "enterprise," while really meaning that one is leading software development teams, not matching up the business strategies/process, etc. with technology at whatever level of abstraction is necessary to carry out or analyze the architecture.
Whether organizations are looking for a full-time employee or a consultant/contractor, the clearer the position is defined, the greater the chance of success in the vetting and hiring processes. Before coming up with the interview questions, the role/job needs to be defined, hashed out with management and peers, and then the interview approach and questions can be formulated. If certain parties expectations are unrealistic, the time to reconcile that is not during candidate interviews, but during the formulation process defining the position. Otherwise, the potential for wasting a lot of people's time, including those interviewed, is high.
A few tips about the position formulation process:
- Stop looking for Superman or Superwoman. Everybody always wants a type-A, all-star performer than can do business requirements, data architecture, project lead/management, integration, coding, web design, and fill in for the database administrators as needed. We all know that this is unrealistic, so why are all of these skills being specified in the job requirements? I know very few people who can do all of that well, much less in an 8-10-hour day, and the ones that can are taken and make much more than you're probably willing to pay them. This again begs the issue: what exactly are you looking for this person to do for you?
- If you decide that the position skews more toward the business (i.e. requirements, stories, use cases, higher abstraction-level, etc.), then focusing the interview primarily on the complexities of your technologies isn't going to help you evaluate the candidates that come before you. The focus needs to be more toward the business-technology interfaces you have in place and what the candidate must accomplish for you in that respect. The best candidates in this situation are also very familiar with your industry, such as those that have worked for your competitors or peer organizations.
- Conversely to (2), if you define the role as more technical/team-lead in nature, then the opposite is relevant and your technical details, projects underway, and other issues along those lines are in play. The questions and discourse here are more specific project-oriented. I like to pose a project-related problem to a candidate and have her articulate her experience along those lines, with some questions that draw out her command of technical details to a point. Arguing technical minutiae or flaws with job candidates is a waste of time. If the candidate makes mistakes, note it to yourself so you can refer back to it later when your organization makes its decision. You are evaluating a candidate for a job, not arguing with the candidate over the correct answer or approach - that comes later after you've hired them...:)
- Make very, very certain that you screen for what I'll call 'motivation and personality' in addition to technical and business skills. Superman may be before you at the interview table and give you every answer you want to hear. He may also be a narcissistic prima-donna that will have your staff in revolt within a week after he starts. How this gets handled is a wide-open topic in itself, but here's a short synopsis of what I focus on: working with and communicating to others; dealing with setbacks; work ethic; political skills/savvy'; and work organization.
Organized and purposeful job descriptions help in quite a few ways, with the primary benefit of HR and recruiters getting the right candidates in front of you or on the phone, with the additional benefits of saving a lot of time and effort of everyone involved and ecentually making the right decisions on who to bring on-board.