You've reached the internet home of Chris Sells, who has a long history as a contributing member of the Windows developer community. He enjoys long walks on the beach and various computer technologies.
Friday, Aug 1, 2014, 3:09 PM in Interview
Future Proof Your Technical Interviewing Process: Hiring or Not
This is the last in a 4-part series on how to interview well. Parts 1-3 covered the phone screen, the technical interview and the fit interviews. In his part, we’ll wrap up by talking about how to make the hiring decision.
Make Time For Questions
As important as what questions you ask the candidate are leaving time for them to ask their questions. Remember that they're interviewing you, too. Be open and honest about the answers; technical people have a sensitive bullshit detector, so don't try to pretend that everything is perfect; they’ll know if you’re not being sincere. However, it’s a fine line. If you find yourself dwelling on the negative, you have to wonder if you've found a good fit for yourself.
Also, don't forget to factor their questions into your own thinking about the candidate. The questions they ask about a job and a team they're going to be spending 40+ hours/week with is as good an indicator of how they think as anything else.
Making the Call
As you pass the interview candidate from person to person, make sure that you spend a few minutes in private with the next interviewer talking about what you heard that you liked as well as things you'd like them to circle back on. You want to give them an opportunity to try again, either to convince you it's not an issue or to confirm that it is.
Every interviewer should share their thoughts about the candidate soon while they're fresh. You can send an email around to the team as you finish or get together in the same room after the candidate has headed home, but it should be the same day; those first impressions matter.
Ultimately each interviewer will provide three pieces of information: a thumbs up/down (whether you use actual thumbs for this process is up to you : ), a confidence level (do you really love this person? are you on the fence?) and an explanation (“I loved how they think about the customer!” or “They never figured out how to efficiently search an infinite space of possible solutions.”)
The set of interview results will come out in three ways:
- Everyone loved that candidate. Hire them.
- Everyone hated the candidate. Don't hire them. Be polite!
- There's a mix. Discuss. Potentially get more info.
Of course, options #1 and #2 are easy to deal with. Unfortunately, option #3 is where most candidates fall. The question is, what do you do with a candidate with mixed results? If you're following the principle that it's better to send a good candidate away then to hire a bad candidate, then you'll pass on them. However, you'll want to spend some extra time on candidates like these. Discuss it amongst the team. See how adamant the thumbs up voters are and why. See how adamant the thumbs down voters are and why. If the candidate is on the fence but leaning towards "hire," pick someone else to talk to them and/or get them into a different environment, e.g. the bar down the street or the bowling alley at the company Xmas party, and see how they do.
Ultimately, it boils down to one thing: does the team as a whole want to bring the candidate into the team? If so, great. If not, let them go. Certainly a senior member of the company or department can override the team and hire a candidate above their objections, but I wouldn't recommend it. You're much more likely to hurt a good team in those situations then to help it.
Where Are We?
Whether you agree with the specifics of this process or not, I encourage you to spend the time to really examine your process. You want the team you build to be more than the sum of the parts, but that kind of magic requires first that you have great parts.
Tuesday, Jul 29, 2014, 4:19 PM in Interview
Future Proof Your Technical Interviewing Process: The Fit Interviews
If you just found yourself here, you’ve stumbled onto a multi-part series on the technical interviewing process. Part 1 covered the phone screen and part 2 covered the technical interview. Today we’re going to discuss the “fit” interviews, that is, team and cultural fit.
The Team Fit Interview
Modern software development is done in teams. You want to be able to judge any candidate as a productive, positive member of your team. They don't necessarily have to have experience doing things the way you do them, but they should show the ability to adapt when issues arise. Your job in the team fit interview is to break the important things that happen in your team into situations that you can ask your candidate about. The following are pretty standard examples:
- What's the right process for gathering requirements?
- How do you convince someone that you've got a good idea?
- What do you do when you can't convince them?
- How do you deal with vague requirements?
- What happens if you're asked to do something you don't agree with?
- etc.
However, you have to be careful here. Pretty much anyone can give you the "right" answers to these questions, but you don't want the "right" answers – you want the real answers. How does a candidate actually behave in the face of these situations?
The best way I know of to get the real answers out of someone is something called Behavioral Interviewing. The idea is simple: instead of asking someone how they would act if faced with a certain situation, ask them to describe an example in their past when they've had to deal with that situation. Discuss it with them. How did their strategy work for them? What did they learn? What would they do differently?
Just this one shift from “how would you deal with this situation” to “how did you deal with this situation” will get you a much deeper look into how a candidate actually behaves, which allows you to decide if they're a good fit for your team.
The Cultural Fit Interview
This goal of the cultural fit interview is to figure out if the candidate will like their new working environment and whether the team will be glad to have them. It's enormously important and very difficult to access. One typical way to approach this type of interview is to ask the following kinds of questions, also in a Behavioral Interviewing style:
- (You're a startup) How do you like the idea of quick decisions, hard work on short deadlines, light process and tight purse strings?
- (You're an established company) How do you like the idea of getting buy-in with a set of stakeholders, making sure we don't ship anything until it's done, following an established process and sticking to a budget?
- What's more important: the customers or the business?
- What kinds of activities are most important to you? Do you like to be focused on your set of tasks or do you like to do a lot of different things?
- What makes you as productive as you can be?
- Where do you see your career path taking you?
- etc.
These questions are much more vague and really meant to start a conversation, but they're also very hit-and-miss. If you happen to hit the right path, you can really crack a candidate open like a ripe nut.
Also, you want to be careful how you interpret the answers. If you don't filter out people that aren't a good fit for the culture of the company, they'll be unhappy and you'll be unhappy. On the other hand, if you filter too much, you'll lose out on the benefits of diversity. It's a hard line to walk.
Another way to approach a culture fit interview is to get creative. Maybe invite the person to a company event, perhaps a semi-public mixer or a Friday afternoon beer bash. Maybe sit down with the team over lunch and play a game together. Maybe sit in the café and grab lunch in a small group and see how the conversation goes.
I think the key to finding a good fit culturally is to spend time with the candidate that doesn’t center around the technology you’re using to build your products. For example, involving a candidate in something that the team does for fun can go a long way towards finding a great new member for your team.
Next Time
Tune in next time for when we wrap this series up and talk about how to make the hiring decision.
Thursday, Jul 24, 2014, 12:46 AM in Interview
Future Proof Your Technical Interviewing Process: The Technical Interview
It’s incredibly important to interview well as you’re building your technical team. Further, interviewing well is hard to do and, like anything, you only get out of it what you put into it. In part 1 of this series, we discussed the phone screen. In this part, we’ll discuss the technical interview.
The Technical Interview
The only way to really know if someone can deliver technically is to give them a problem to solve and watch them solve it. You can do this with simple data structure problems on the whiteboard, test questions on paper, algorithm problems in notepad, real-world problems with some pair programming or puzzle problems with them waving their hands wildly in the air. In a technical interview, you should encourage the candidate to think out loud, because you care more about how they go about solving the problem then actually getting to an answer. You will look for the following things:
- Are they asking questions to solidify vague requirements?
- Are they approaching the problem from a logical angle (even if it's different from what you had in mind)? Does problem solving come naturally to them?
- Are they making the right use of the features available in whatever sandbox you gave them in which to solve the problem, e.g. making good use of list comprehensions in Python?
- Are they writing good code and pointing out the shortcuts they're taking due to the medium they're using, e.g. the whiteboard? Does the coding come naturally to them or are they struggling?
- Do they come up with a reasonable answer? Are they on the right path even if they run out of time?
- Do they understand the "why" behind their answer as well as the "how?"
This last one is the one I tend to focus on the most. Even more important than a candidate having knowledge of the technologies you're going to ask them to use is their ability to understand new technologies over time.
My father always says that while teenage drivers hopped up on testosterone may get into the most accidents, they're the ones that push the cars to see what they will do. You want to hire engineers that have pushed technologies past their limits for the pure joy of it. Those are going to be the ones that build the deep knowledge and can adapt in the future to whatever comes their way.
I filter for deep understanding by not just digging into not only the "how" of whatever they claim to know best, but also the "why." They may know how to build a factory in Angular, but do they understand what a factory is and why Angular does it that way? They may know how to manage their resources in the face of the JVM's garbage collector, but do they know why we use garbage collection and what the downsides are? Do they understand what canvas is good for, what SVG is good for and when to choose which?
The key here is that past behavior indicates future behavior – if they're developed deep understanding of the technologies they've learned before, chances are pretty good that they're going to be able to do that for the new technologies your team adopts in the future. There is no better way to understand how well they’re going to do on future technical challenges than hearing how they’ve handled such challenges in the past and seeing how they do it right in front of you.
What’s Next in This Series
However, the technical fit is not the only thing you need to look for – you also want to make sure that they will fit in well on your team and the company culture overall. We’ll talk about these in the next piece in this series.
Monday, Jul 21, 2014, 9:08 PM in The Spout Interview
Future Proof Your Technical Interviewing Process: The Phone Screen
In 30 years, I've done a lot of interviewing from both sides of the table. Because of my chosen profession, my interviewing has been for technical positions, e.g. designers, QA, support, docs, etc., but mostly for developers and program managers, both of which need to understand a system at the code level (actually, I think VPs and CTOs need to understand a system at the code level, too, but the interview process for those kinds of people is a superset of what I'll be discussing in this series).
In this discussion, I'm going to assume you've got a team doing the interview, not just a person. Technical people need to work well in teams and you should have 3-4 people in the interview cycle when you're picking someone to join the team.
The Most Important Thing!
Let me state another assumption: you care about building your team as much as you care about building your products. Apps come and go, but a functional team is something you want to cherish forever (if you can). If you just want to hire someone to fill a chair, then what I'm about to describe is not for you.
The principle I pull from this assumption is this: it's better to let a good candidate go then to hire a bad one.
A bad hire can do more harm than a good hire can repair. Turning down a "pretty good" candidate is the hardest part of any good interview process, but this one principle is going to save you more heartache than any other.
The Phone Screen
So, with these assumptions in mind, the first thing you always want to do when you've got a candidate is to have someone you trust do a quick phone screen, e.g. 30 minutes. This can be an HR person or someone that knows the culture of the company and the kind of people you're looking for. A phone screen has only one goal: to avoid wasting the team's time. If there's anything that's an obvious mismatch, e.g. you require real web development experience, but the phone screen reveals that the candidate really doesn’t, then you say "thanks very much" and move on to the next person.
If it's hard to get a person to come into your office -- maybe they're in a different city -- you'll also want to add another 30 minutes to do a technical phone screen, too, e.g.
- Describe the last app they built with Angular.
- Tell me how JVM garbage collection works.
- What’s the right data structure to hold the possible solutions to tic-tac-toe?
Whatever it is, you want to make reasonably sure that they're going to be able to keep up with their duties technically before you bring them on site, or you’re just wasting the team’s time.
At this point, if you're hiring a contractor, you may be done. Contractors are generally easy to fire, so you can bring them on and let them go easily. Some companies start all of their technical hires as contractors first for a period of 30-90 days and only hire them if that works out.
If you’re interviewing for an FTE position, once they’ve passed the phone screen, you're going to bring them into the office.
You should take a candidate visit seriously; you're looking for a new family member. Even before they show up, you make sure you have a representative sample of the team in the candidate's interview schedule. At the very least, you need to make sure that you have someone to drill into their technical abilities, someone to deal with their ability to deliver as part of a team and someone to make sure that they're going to be a cultural fit with the company as a whole. Each of these interview types is different and deserves it's own description.
Future Posts in This Series
Tune in to future posts in this series where we’ll be discussing:
Friday, Jul 18, 2014, 9:49 PM in Interview
Head of Google interviewing says “results matter, riddles don’t”
“[B]ainteasers are a complete waste of time. How many golf balls can you fit into an airplane? How many gas stations in Manhattan? A complete waste of time. They don’t predict anything. They serve primarily to make the interviewer feel smart.”
In another interview, Bock said that when putting together a resume, focus on what you did in relation to the expectations:
“The key is to frame your strengths as: ‘I accomplished X, relative to Y, by doing Z.’ Most people would write a résumé like this: ‘Wrote editorials for The New York Times.’ Better would be to say: ‘Had 50 op-eds published compared to average of 6 by most op-ed [writers] as a result of providing deep insight into the following area for three years.’ Most people don’t put the right content on their résumés.”
Amen!
Friday, Jun 18, 2010, 11:43 AM in Interview
David Ramel Asks About Interviewing at Microsoft
David Ramel from 1105media.com is writing an article that includes the Microsoft interviewing process and he send me some questions:
[David] How would you succinctly sum up the Microsoft interview process as compared to those of other tech companies?
[Chris] MS does some things similarly to other high-tech companies I've worked with, e.g. having each interviewer focus on an aspect or aspects, e.g. team skills, people skills, technical skills, etc., expecting a candidate to ask questions, communicating between interviewers to push more on one area or another, etc. The riddle questions are a uniqueness at Microsoft (at least they were when I last interviewed), but theyire pretty rare these days. Coding on the whiteboard also seems pretty unique to Microsoft (myself, I prefer the keyboard : ).
[David] How has the Microsoft interview process changed over time? (Microsoft seems to have shaken up the tech interview process some years ago with those brain-teasing puzzle� questions, but now seem to be much more technically-oriented and job-specific. Just wondering about your thoughts on this observation.)
[Chris] While I have had them, puzzle questions were rare even when I was interviewed 7 years ago. Since then, I haven't run into many people that use them. However, when they are used, an interviewer is often looking for how a candidate works through an issue as much as the solution that they come up with. In an ever changing world, being able to learn and adapt quickly is a huge part of how successfully you can be in the tech industry at all and at Microsoft specifically. I prefer technical design questions for these kinds of results, however, and it seems that most 'softies agree.
[David] What would you say was the biggest factor in your being offered a job at Microsoft?
[Chris] I had a reputation outside of MS before I interviewed, but that almost didn't matter. If I hadn't done well during the interview, I would not have been offered the job. When in doubt, a team generally prefers to turn away a good candidate rather than to risk taking on a bad one, so if there's anything wrong, team fit, technical ability, role fit, etc., a candidate won't get an offer.
[David] What's the single most important piece of advice you can offer to those preparing for a Microsoft job interview?
[Chris] You asked for just one, but I'm going to give you two anyway. : )
- If you need more information to answer a question, ask for it. Thatis how the real-world works and many questions are intentionally vague to simulate just this kind of interaction.
- Try to answer non-technical questions based on your personal experience, e.g. instead of saying "here's how I would deal with that situation,"� say "I had a similar situation in my past and hereis how I dealt with it."� This is a style of interviewing known as behavioral� and even if your interviewer doesn't phrase his questions in that way, e.g. "give me an example of how you dealt with a situation like blah,"� it's helpful and impressive if you can use your own history to pull out a positive result.
[David] Could you please share any other observations you have on the Microsoft interview process that may not be covered in your site or the Jobsblog?
[Chris] I run a little section of my web site dedicated to the MS interviewing process and the thing I will tell you is this: don't prepare. Be yourself. If you're not a fit for MS, no amount of preparation in the days before an interview will help and if you are a fit, that will come through in the interview. Also, make sure you ask questions. Working at Microsoft isn't just a job, it's a way of life, so make sure you're sure you want the team and the job for which you're interviewing.
[David] Does MS provide training for interviewers? If so, what do they stress most?
[Chris] I'm sure MS does provide training for interviewing, but Iive never been to it. At Intel, I learned the behavioral interviewing technique, which Iive used in every interview since, both as an interviewer and as a job candidate.
[David] Do you have standard questions, or do you tailor them to the situation? If the latter, is it usually tailored for team fit, to a specific open position, particular skills, etc.?
[Chris] I have once standard technique question and a few standard behavioral interview questions. The technical question is to ask them what their favorite technology is and/or what they consider themselves to be an expert� in and then drill in on their understanding. If they can answer my questions deeply, this shows passion about technology and the ability to learn something well, both of which are crucial for success at MS.
My behavioral interviewing questions are things like "Tell me about a time when youive been in conflict with a peer. How did you resolve it? What was the result? What did you learn?"� and "Tell me about a time when you had much too much work to do in the time you were given. How do you resolve that issue? What was the result? What did you learn?"� The core idea of behavioral interviewing is that past behavior indicates future behavior, so instead of asking people things like "How would you deal with such-and-such?�" you ask them "How did you dealt with such-and-such in the past?"� This forces them to find a matching scenario and you get to see if they way they dealt with the issue in real life matches what you want from a team mate in that job.
[David] How would you describe the kinds of coding questions you ask? A couple of real examples would be perfect!
[Chris] I don't often ask coding questions, but when I have, I let them use a keyboard. I hate coding on the board myself as it's not representative of how people actually code, so I don't find it to be a good indicator of what people will actually do. I guess I even use behavioral techniques for technical questions, now that I think about it. : )
Tuesday, Jul 25, 2006, 10:52 PM in Interview
Ed Helms on Microsoft Recruiting
This spoof on Microsoft's college recruiting practices was recorded long ago (back with the XBox was new), but it has recently surfaced again, so I thought I'd share. Enjoy.
Wednesday, Nov 23, 2005, 9:43 AM in Interview
Wasting the Prince of Darkness
From "Pete" (not his real name):
I walked into my first technical interview at Microsoft, and before I could say anything, the woman says, Youre in an 8x8 stone corridor. I blink and sit down.
Interviewer: The prince of darkness appears before you.
Me: You mean, like, the devil?
Interviewer: Any prince of darkness will do.
Me: Ok.
Interviewer: What do you do?
Me: <pause> Can I run?
Interviewer: Do you want to run?
Me: Hmm I guess not Do I have a weapon?
Interviewer: What kind of weapon do you want?
Me: Um something with range?
Interviewer: Like what?
Me: Uh a crossbow?
Interviewer: What kind of ammo do you have?
Me: <long pause> Ice arrows?
Interviewer: Why?
Me: <floundering> Because the prince of darkness is a creature made of fire???
Interviewer: Fine so what do you do next?
Me: I shoot him?
Interviewer: No what do you do?
Me: <blank stare>
Interviewer: You WASTE him! You *WASTE* the prince of darkness!!
Me: <completely freaked out and off my game> Holy crap what have I gotten myself into.
She then tells me that she asks that question for two reasons. 1) Because she wants to know if the candidate is a gamer (which is apparently really important please note: Im not a gamer) and 2) because she wants her question to show up on some website. I hate to accommodate her, but this is definitely the weirdest interview question Ive ever heard of.
Well, here you go, weird-prince-of-darkness-wasting-lady...
Tuesday, Sep 6, 2005, 2:29 PM in Interview
Stumped
Tue, 9/6/05, 2:29 pm
Tuesday, Feb 22, 2005, 12:30 PM in Interview
Scott Hanselman's Great .NET Developer Questions
Scott Hanselman has posted a set of questions that he thinks "great" .NET developers should be able to answer in an interview. He even splits it up into various categories, including:
- Everyone who writes code
- Mid-Level .NET Developer
- Senior Developers/Architects
- C# Component Developers
- ASP.NET (UI) Developers
- Developers using XML
Am I the only one that skipped ahead to "Senior Developers/Architects" to see if I could cut Scott's mustard?
Friday, Jan 21, 2005, 8:16 PM in Interview
Jason Olson's Microsoft Interview Advice
Jason Olson recently interviewed for an SDE/T position (Software Development Engineer in Test) at Microsoft and although he didn't get it, he provides the following words of advice for folks about to interview for the first time:
- Just Do It
- Remember, no matter how much you might know your interviewer, it is important to not forget that it is still in interview
- Pseudocode! Pseudocode! Pseudocode!
- But, as long as you verbalize what you're thinking you should be in pretty good shape
- Bring an energy bar or something to snack on between breaks in order to keep your energy level up
- [B]ring a bottle of water and keep it filled up
- A lunch interview is still an interview!
- Know the position you're interviewing for [ed: unless you're interviewing for an editing position, in which case you should know the position for which you're interviewing]
- Your interview day is not only your opportunity to be interviewed, but also your opportunity to interview the team/company
You can read the full story on his web site.
Sunday, Aug 22, 2004, 8:22 AM in Interview
Standing Out When Submitting Your Resume
After seeing all of those pictures in Wired of the wacky letters that people send, I love the idea of Michael Swanson opening the floodgates by sending his resume along with a life-size cardboard figure. What's next?
Friday, Aug 20, 2004, 3:22 AM in Interview
Some of the MS Interview Process Filmed (Finally!)
Channel9 did what I was unable to ever get done: filmed some of the interview process (part 1, part 2 and part 3). It's not an actual interview, but Gretchen Ledgard and Zoe Goldring, both Central Sourcing Consultants at HR for MS, lead you through what to expect at a Microsoft interview, providing a wealth of wonderful tips, e.g.
- MS is casual, so it doesn't matter so much what you where (i.e. don't feel you have to wear a suit, but don't show up in flip-flops and headphones around your neck [still playing!]). Regardless of what you where, it's what's in your head that's important.
- Interact a lot of with the interviewer. Ask questions, think out loud, etc. The questions are meant to be vague and again, it's about what's going on in your head, so verbalize it.
- Bring water if you're thirsty, not coffee, as spilling coffee is going to leave a much more lasting stain/impression.
- MS rarely asks logic/riddle questions anymore. They're not a good indicator of a good employee.
- Expect coding questions if you're a dev, testing questions if you're a tester and passion questions no matter what.
- If an MS recruiter calls, don't expect them to have a specific job in mind. Instead, expect to be asked what you'd like to do at MS.
- If the first interview doesn't take, it may well be that you're right for MS but not right for that job. It can literally take years to find the right job for you at MS.
BTW, I have to say that I never got a ride on an HR shuttle. I guess they save that for the "good" hires... : )
Tuesday, Jul 20, 2004, 4:19 PM in Interview
Questions for Testers
A friend of mine sent along some questions he was asked for a SDE/T position at Microsoft (Software Design Engineer in Test):
- "How would you deal with changes being made a week or so before the ship date?
- "How would you deal with a bug that no one wants to fix? Both the SDE and his lead have said they won't fix it.
- "Write a function that counts the number of primes in the range [1-N]. Write the test cases for this function.
- "Given a MAKEFILE (yeah a makefile), design the data structure that a parser would create and then write code that iterates over that data structure executing commands if needed.
- "Write a function that inserts an integer into a linked list in ascending order. Write the test cases for this function.
- "Test the save dialog in Notepad. (This was the question I enjoyed the most).
- "Write the InStr function. Write the test cases for this function.
- "Write a function that will return the number of days in a month (no using System.DateTime).
- "You have 3 jars. Each jar has a label on it: white, black, or white&black. You have 3 sets of marbles: white, black, and white&black. One set is stored in one jar. The labels on the jars are guaranteed to be incorrect (i.e. white will not contain white). Which jar would you choose from to give you the best chances of identifying the which set of marbles in is in which jar.
- "Why do you want to work for Microsoft.
- "Write the test cases for a vending machine.
"Those were the questions I was asked. I had a lot of discussions about how to handle situations. Such as a tester is focused on one part of an SDK. During triage it was determined that that portion of the SDK was not on the critical path, and the tester was needed elsewhere. But the tester continued to test that portion because it is his baby. How would you get him to stop testing that portion and work on what needs to be worked on?
"Other situations came up like arranging tests into the different testing buckets (functional, stress, perf, etc.)."
Tuesday, Jul 6, 2004, 12:20 PM in Interview
What do job interviews really tell us?
For the New Yorker, Malcolm Gladwell discusses various indicators of how well interviews actually work for screening job candidates (in a phrase: not very well). The discussion of how we make our decision about someone in the first 2 seconds after seeing them and then use our future interactions with them to either reinforce our initial reaction or forgive as an aberration is particularly telling.
22 older posts No newer posts