Twitter Summary: Improve the quality and the outcomes of all your meetings through the use of Openspace Technology and its principles.
I was introduced to Openspace Technology (OST) at Amazon.com and have used it at subsequent organizations with great success. OST is not “technology” as in gadgets that surround us. It is a process to create meetings that takes minimal work from organizers, and pushes the responsibility for success to the participants who are encouraged to take ownership and solve the issues. The best summaries of OST are the wikipedia entry and the user’s guide hosted at OpenspaceWorld.org.
Rather than highlight how OST works, which is already detailed in the above links, I want to highlight the impact on the participants. It can best be described in there phases:
- Introducing OST — When first introducing Openspace Technology to a group of people, I typically see incredulity that this could be a successful way to run a meeting. The expectation from other presenters that it can’t be successful without having a detailed agenda and preparing powerpoint slides to lead the group. The expectation from attendees is that it is too fluid to capture what was really important. Fortunately, those impression change once everyone realizes that this is an opportunity to contribute their critical issues. By stating aloud what they think is important, they can hear other people come up with issues and help determine the relative priorities.
- Meeting Sessions — The self-organizing nature of the meeting becomes apparent as everyone offers session ideas. As ideas are contributed, they spawn more ideas that are added to the pool of potential discussions. As more important issues are made known, some ideas are tabled for future discussions. The impact on the group is they realize that with the limited number of people in the room, they either need to take ownership of an idea, or contribute to an initiative they believe will make the group successful. The participants become energized as the process makes it clear that some issues won’t be resolved unless someone (a) states there is an issue, (b) takes ownership of fixing that issue, and (c) finds others who think the issue is important and can help resolve it.
- After-effects — The after-effects of running an OST meeting can be great and frustrating. The impacts of running a meeting is that it energizes the participates, distributes ownership, and creates buzz around the important issues. Really important issues get lots of support and can enlist help from a broad base. Frustrations that emerge stem from either (a) having a long list of remaining tasks that may not get addressed, and (b) the team feeling the company is not pursuing the priorities as generated from the meeting. Frustration can be managed by capturing and prioritizing the future work and ideas generated at the meeting.
The principles of OST work in smaller non Openspace meetings. I often would run my weekly engineering meetings much the same way. I would send out a minimal or blank agenda prior to the meeting, soliciting the team for topics that they wanted to share in the group setting. If a team member knew of a topic that should be discussed, there was a regular forum where they had the responsibility to surface issues. If we had an empty or light agenda the meeting would be skipped or shortened. If the agenda was full we would need to decide what was important for that week and re-schedule other issues for another time.
Having a shortage of time or resources is a fact of life in most organizations. Using OST prinicples to manage the list of things to do helps focus on getting the most important things done in the time alotted.
Twitter Summary: Some of your most important performance reviews don’t happen at work.
Performance reviews come from everyone in your life, its just that work is the only place where they have formalized it and based your compensation on its result. I frequently joke that every year my family should give me a “How am I doing?” customer service card.
Many years ago I was approached by a colleague who was concerned that his performance review at work for the year would not be up to his previous year’s reviews. He was intending to take the maximum amount of time he could take for paternity leave, but also wanted to keep his standing as one of the top engineers in the company. His concern was how would he be able to maintain his high ranking while still keeping his commitments at home. I was about to offer a suggestion to help, but realized that the best advice was to make sure when he went on paternity leave, he actually took the time off.
Your work performance review is important enough to rate the time and energy to create a formal process to collect information, assess performance relative to others, and make recommendations for future improvement. Unfortunately, a 360 review encompasses only your performance at work, it doesn’t cover your performance as a father, friend or spouse. I imagine our perspective about the importance of our yearly work reviews would be much different if the 360 review included a) your kids review your abilities as a parent, b) your friends rating of how much their lives are made better with you in their life and c) your significant other reivews your contribution to their life.
Twitter Summary: Short-term focused performance reviews can help you deliver on tangible results.
I have a personal loathing for the yearly multi-page performance review where: A) All an employee’s past year’s work is reviewed, B) All their work for the upcoming year is planned and mapped to company, organization, and team objectives, C) All their personal development tasks for the year are listed, and D) 360 review feedback is summarized, E) Everything is distilled to a single letter/number/score, and F) Becomes the basis of the employee’s compensation, bonuses and future path within the company. The process is difficult and drawn out for the managers who have to assemble the information for their team. It can take weeks to assemble the information and craft a review and schedule a meeting for each employee. The process is also difficult and drawn out for the employee who has to wait until everything is assembled to find out how the company thought they performed this year. This is further complicated by the opacity in what is going on with the reviews as it is taking so much time to do. The management gurus at Harvard Business School highlight that performance reviews are imperfect and are looking for better ways to improve the process suggesting that the best way is “continuous performance reviews”.
In startups and agile organizations this approach is especially problematic as the goals for the team and company can change over the span of a few weeks. Rewriting documents for each employee so that they can have the benefit of an accurate assessment for the employee’s performance review adds even more time to organizations that are striving to be nimble.
The best process I have seen to date has been to a) de-couple all the reviews bundled into the yearly review and spread them out through the year and b) set the employees expectations as to the goals of each review element and how it impacts their career growth. The types of performance reviews that should be split throughout the year are:
- Company and team performance reviews can happen quarterly or bi-anually so that management can set expectations if the company is doing great, or if it is struggling to keep payroll and what everyone can expect in terms of compensation and future tasks.
- Employee performance reviews that happen quarterly can cover what happened last quarter and what needs to happen in the next quarter for the employee and the company to be successful
- Personal development reviews can happen 2 to 4 times per year covering what the employee would like to learn to do their job better and what the team would like to help deliver for the customer.
- Compensation reviews can happen 1 to 2 times per year, on a separate cycle where all the previous performance and personal reviews for the past year are assembled and the employee’s contributions and the impact of the team’ and company’s work can help reflect what types of compensation adjustment are necessary.
By splitting out all the reviews into smaller more frequent reviews you gain the benefits of:
- Less work for managers and employees who can focus on deliverables for the next quarter.
- Smaller focused reviews are easier for managers and employees to write-up and explain.
- Manager and employees have more frequent checkins as to employee’s performance in the context of the team and the company.
- Personal development tasks are given priority as it is not hindered by people thinking about their performance or compensation bundled in the same document.
- Decoupling the compensaton conversation from multiple performance evaluations helps the manager set the employee’s expectaions as to how much of an adjustment can be made.
The role of performance reviews is important in helping make sure that the team is focused on the correct things for the company to deliver on its goals. By setting the employees, managers and company’s expectations for each performance review element covers, you increase the transparency in how the company will succeed. This advice does take some extra scheduling steps for managers, but spreading the tasks throughout the year ensures that there is constant communication of managers with employees. In addition anything that takes a company away from a 20 page per employee review is likely to be less onerous, and help move towards a clearer and better run performance process and happier employees.
Twitter Summary: Recognizing individual contributions with artifacts of success are a part of any team culture. Make sure to do it right.
Physical artifacts of success have been around as long as people have been cooperating in accomplishing goals. Hunters collect trophies of their first kills. Microsoft programmers get “ShipIt” awards. These physical mementos are typically very group sensitive and can be very elaborate especially when you compare the United States Military awards and decorations to the simpler but more permanent Maori tattoos. Despite the differences, their purpose is so that groups can identify their own and have an idea of how each has contributed to the benefit of their group.
Most companies I have worked for has had an analogue to this from t-shirts and laminated sheets of paper for product launches to Lucite plaques for creating patents. In the best of cases, there are pretty clear requirements for awards. To get a patent block, you must have a patent application. In the worst case, it can feel like the mementos are just being handed out to anyone who is standing nearby or to curry favor. This can cause resentment among the team members and create disincentives among the very people who you want to show your appreciation.
With the potential issues surrounding physical artifacts of success, it is tempting just not to create any at all. However, every group develops its own symbols to self-identify. If you can come up with a clear simple rule set for how these artifacts are given out, you can maintain the integrity of the mementos and give people recognition for how their efforts have helped the team.
Qualities for creating a meaningful physical “Artifact of Success”:
- Goal based — Give “artifacts of success” only for contributions that help with company’s goals. Product, project launches that create substantive value and took focused organization effort for the company should be recognized. Don’t recognize things that are nice, but not distinguishing for the company.
- Limited — The mementos will only be created for the few that achieved the goal or contributed for the benefit of the team. If necessary, create a simple rule to identify who gets things and who doesn’t. Don’t devalue the identification by handing them out arbitrarily.
- Inexpensive — By using inexpensive tokens (paper, baseballs, golf balls, origami birds, colored pins), there is no mistaking that the purpose of this is recognition not compensation.
- Portable — As much as I love a good t-shirt, I can’t carry all of them with me, so having a jar of launch golf balls, or tokens that I can post on a wall or hold on a desk will suffice.
- Sincere — I am always a fan of the memento listing: The name of the event, the date of the event, and being cross-signed by all the team-members at the launch party, or by the CEO/organizational lead for the company.
- Composable, stackable, groupable — If the physical artifacts of success can be composed, similar to military badges, Maori tatoos or Amazon’s patent blocks, you create a compelling visual effect that enhances the value when you have sets of successes.
By spending a little time on what qualifies as something that deserves “recognition” and what would be team appropriate, you should be able to create your own visual shorthand to recognize team members contributions to the company.
Twitter Summary: A great hire will often generate lots of energy during a hiring meeting and some of it negative.
Once you have established whether a candidate passed your interview, the final step is to have a hiring meeting where all the interviewers provide their assessment of the candidate. The hiring meeting is important to attend in-person for multiple reasons: a) You are compelled to make a decision as you have a scheduled time and duration to decide on a candidate, b) You can confirm or provide counter-evidence for skills or behaviors people saw during their interviews, and c) You can calibrate with your team whether you are all asking the right questions with the right degree of difficulty. Email doesn’t work in this context as in-person dialogue allows the team to quickly clarify questions among all interviewers.
How do you decide if you are going to hire someone? The best technique I have seen to date has been requiring unanimous consensus by all the interviewers that the candidate should be hired. The tremendous flaw of this style of decision making is that it will reject candidates who could have done well at the company, but because of a sub-par interview, they are eliminated from consideration. The incredible advantage of consensus driven hiring is that everyone who is hired has had at least six people say “Yes” to hiring the candidate. If all of your colleagues have gone through the same process to join the company you should have more confidence that they are doing the right thing. If a co-worker begins having difficulties, there should be a sense of ownership by the team that they picked the candidate for a reason and they can help them work through those issues.
It is important to note, that consensus means that everyone agrees the candidate should be hired. It does not mean that everyone likes the candidate. Some of the best employees I have ever hired had at least one or two other interviewers indicate they were not inclined to hire. There have been cases where a college intern was determined to be a great hire by two of their interviewers, and a strong no-hire, by their other two interviewers. This intern was fortunately hired and ended up being an exemplary hire for the company as the advanced and took on more projects quickly. It was fortunate that in these cases the candidate was able to express a deep technical capability and being a “superstar” in some fashion that an interviewer was willing to champion them in the hiring meeting to take a chance on a hire. The dissenting interviewers shouldn’t ever be bullied into hiring a poor candidate, but they should be open to listening to the interviews of others and decide if it is worth the risk of sharing a payroll with the new hire.
Some of the most questionable hires have been hiring meetings where the candidate did well enough that everyone was inclined to hire, but no interviewer was able to figure out where the candidate was great. The problem with hiring these candidates is that they frequently have a similar career within the company of doing good but not great. Finding a passion or the superlative associated with each candidate means that if they decide to join the company you will increase the overall depth of your company’s employees and the probability that something great will emerge.
Twitter Summary: The questions you pose to interviewing candidates reveals if your job is interesting or boring.
The questions you ask and your assessment of the candidates answers comprise the core of the interview. More importantly, excellent candidates will be paying attention to you and will be assessing you just as much as you are assessing them. If the questions you ask are the same the candidate has heard in other interviews, seen in interviewing manuals, or from another interviewer that day, they will have a clue that your interviews are not organized, your questions aren’t relevant, and that you may not be capable of capturing the depth and breadth of their capabilities.
Great interview questions have the following qualities:
- The problems you ask them to solve should either be ones that you are familiar with or are currently trying to solve yourself. This gives the candidate a clear indication of the type of work they will be asked to perform. This also gives the interviewer the ability to assess how the candidate would solve a problem that is pressing for the organization right now.
- If you are interviewing a software engineer for a kernel development position, ask them to code a scheduler and talk about memory allocation.
- If the candidate is interviewing for a social networking site, ask the candidate to generate code to generate a user to user similarity score.
- If the candidate is interviewing for a marketing position, ask them how they would evaluate marketing options for your current product.
- Don’t ask a “textbook” questions. — For software engineers, this means asking them to implement “atoi()”. For anyone it would mean questions like: “What is your greatest strength?” Your questions should be open enough that the candidate reveals their strengths in the process of describing another problem they solved.
Expandable in depth and breadth
- The questions you ask should be expandable in either breadth or depth. For software engineers, this would mean asking them to take their solution and and scaling to handle millions of users or machines, or alternately asking them to change their solution to handle multiple, media and content data types. For product or marketing candidates, you could ask them how they would modify the programs or projects to deal with if you wanted to handle a larger diversity of customer groups or a broader product line.
- Don’t ask questions with only one right/wrong answer. Asking questions that have only one correct answer is problematic as it doesn’t give the candidate the opportunity to discuss trade offs in choosing and decision making. In the end, you are hiring people to make judgment calls, and are not hiring a computer to give you a rote answer
- Present only the basics of the problem. The interview should be a dialogue, and interviewers should intentionally give less information then the candidate needs to solve the problem. This will get require them to ask you clarifying questions and give you insight as to how they approach their problem solving.
- Don’t spend all your time talking. If you spend most of the time talking you are not getting the information from the candidate you need to make your decision. You can spend more time talking with the candidate after they have answered your question, and especially if they are hired.
Great inteview questions make your job as an interviewer easier . A great set of interview questions will allow you to understand how the candidate would solve the problems you need to solve in your company, how the candidate deals with challenges as the problem expands, and how they communicate and ask for more information in answering the question. It also tells the candidate about what problems they can expect to be solving and if the role is a match for what they want to do.
Twitter Summary — If after interviewing a candidate, you are unable to make a Hire/No Hire decision, you have failed as an interviewer.
Interviewing candidates involves making a judgment call about the person you are interviewing. One thing that makes this difficult is that there are many resources (web based, books, college or career counselors) who can give a candidate the “right” answer for just about any question an interviewer can ask: “What is your greatest weakness?”, “What is your biggest strength?”, “Where do you see yourself in 5 years? 10 years?” Admittedly, offering this information is an attempt to help others who are interviewing answer the questions “well enough,” but often mask what you really want to know about the person. This puts the interviewer into a situation where they will not be able to make a clear decision on a candidate. When candidates answer all the questions “well enough” they run the risk of hiring someone that will only do “well enough” at their company.
My biggest difficulty in interviewing is when I find I personally like the candidate and think they would be great to know outside of work based on shared personal interests that come up in the course of our conversation. However, shared personal interests is not a determining factor as to whether a candidate can match all of the expectations of a role. The sole job as an interviewer is to form an opinion of a candidate and decide whether they are qualified for their role, and will excel in that role for the company.
The keys to success in interviewing a candidate are:
- Be present — Make sure that your phones and computers are off, and you have space where the door is closed. Distractions that pull you away from focusing on your candidate shortchanges the candidate and your opportunity to make a clear decision. As most interviews last only 45 minutes to an hour, your undivided attention is important in assessing the candidate.
- Assess for your “hidden agenda” — We all look for the same attributes for a candidate: Are they smart, punctual, enthusiastic, hard working? You cannot ask those questions directly, as the answer anyone would give to them would be “Yes.” Instead, you should ask questions like: “What was your favorite project?” and in their response you listen to see if the project was successful and required smarts, or enthusiasm, hard work or punctuality to solve.
- Communicate using multiple methods — Require the candidate to communicate using graphs, words, speech and even assess non-verbal communication, regardless of role. As a future employee, the candidate will need to work with many people in the company and needs to be able to communicate to anyone using whatever technique is most important at the time. By assessing the spectrum of the candidates ability, you will hire people who can express themselves and represent well within the company. The non-verbal communication is important to assess the confidence of their responses. If they are not confident in their responses with you during an interview, they will lack confidence when communicating with their colleagues
- Induce Stress — During the interview you should create some conflict that the candidate needs to justify so that you can see how they manage stress. Stress is a fact of life at companies and is important to assess during the interview. The stress doesn’t need to be excessive, but by requiring the candidate to deal with stressors on their feet you can get a good idea of how adaptable they are to the environment. The most appropriate ways I have done this during interviews have been to either (a) repeatedly change the requirements of the system (steadily increasing the scope of the project as they answer each part successfully), or (b) express to the candidate that I think there can be a better solution using technology X, and to justify their use of technology Y. Get the candidate to discuss the tradeoffs and assess the impact of the stress.
- Choose your questions well — The questions you select for a candidate need to be the ones that allow you to make a “Hire” vs “No Hire” decision within the allotted time frame. If you are not asking questions that get you to the point where you can make a hire decision, you are asking the wrong questions. Don’t waste the candidates time or yours by asking questions that don’t help you assess a candidate.
- Look for the superlative — The thing that distinguishes the “Good Enough” candidates from the “Must Hire” candidate is typically a superlative where the candidate was the “Best” at something. Typically this comes up when the candidate mentions they were the “go-to” person for certain areas or solely responsible for certain pieces of technology. Assess why they did so well in those areas and make a determination if that would be important for your company.
- Take Notes — If you interview many people a day, you should write-up your notes immediately afterward so that you don’t confuse candidates and can discuss the candidate during the interview debrief. Notes also help to re-enforce a second pass assessment of the candidate to see if they matched the requirements for your “hidden agenda” or had a “superlative” associated with their prior work experience.
Interviewing takes lots of focused listening, but once you have identified a candidate to interview they deserve the attention you give to them. This attention is how the candidate assesses your non-verbal communication and gives the candidate the confidence that a “hire” or “no-hire” response from you company was the correct decision for you and them.
The next in the series will cover “What” you should ask during an interview.
Twitter Summary: You have more leverage to influence the direction of a company by the people you hire then what you do.
In my career so far, I have interviewed well over 1000 people. I developed a reputation for being good at it which meant I was interviewing software engineers, product managers, engineering managers, VP, CTOs, and even prospective new attorneys which is far out of my core expertise. My job was to assess their ability to be good at their position and to help figure out if they would be successful within the broader context of the company.
I frequently get groans from engineers about how much of their time interviews consume. In fairness, a badly scheduled interview will fragment a developer’s day to be unusable and developer’s want to spend the time doing what they were hired to do. Unfortunately, a well run interviewing effort typically takes multiple hours of time if you have:
- Phone Screen — Assess if there is a match to the position you are looking to fill.
- Pre-planning — Scheduling and assigning interview roles and competencies to assess.
- Interview — Spending the time to figure out if this is someone you would like on your team.
- Debrief — Write up of interview and discussing whether the candidate is a match.
In spite of the time sink, as an employee you should be on as many interview loops as you can handle and make time to schedule. The benefits of interviewing are tremendous to both your quality of life and the future success of your company. The two reasons to do this are:
- You get to decide who works with you! — Never are you more influential in your own work life then when you get to decide which colleagues you will work with daily. That interview process is crucial to figuring out: Is this candidate smart, innovative and adaptive to your environment? Can this candidate help me solve my problem? What about if the problem occurs at 2AM? Will this colleague help me make a right decision and support me, or will the colleague argue with me when I am wrong and explain it in a way I can understand and agree?
- You are helping direct the compass of the company. — The people you hire are the mission statement of the company. By hiring people that not only have a talent for their role, but also have abilities in other domains you get to broaden the reach of what your company can do. If you hire only system kernel engineers, you shouldn’t be surprised if the next innovation is a recommendation to rebuild a new O/S. If you hire framework developers, you will inevitably be requested to approve a new framework that will help develop your product. If you hire a diverse set of talent that can do their jobs, and also innovate in new directions, you will be able to innovate in many new directions. The abilities of the people you hire, will help dictate what will be considered the “core competency” of your company, and which features will be outsourced to different companies.
Yes, interviewing can be time consuming. However, you can assist in planning your schedule so that interviews are less intrusive, because the benefits are immense for your day-to-day work environment and the future success of your company.
Future posts will cover, How to interview, What your interview should cover, When you should interview.
Twitter Summary: When your see a problem that requires you to spend time, money or space. Spend money or space, because time is the only one you can’t reuse.
A classic tradeoff in computer science is the time vs space tradeoff in executing an algorithm. As mentioned in an earlier blog post (An Introduction to Search Engines). If you spend extra disk space storing index data, you can improve the speed of a search by orders of magnitude. The decision here is clear cut. It does take some extra over-head to manage the indices, and update them as the data changes, but the benefits to the customer are immense. The other practical advantage is that as the cost of disk space has plummeted, all the overhead is the management of the data and very little of what was actually stored and computed.
In the startup world, the tradeoff presents itself as “Build” vs “Buy”. Should we build a website from scratch or buy a content management system to display our information? The decision here is ocasionally not as clear-cut and depends on what you want to be the core expertise of your company. If you have a company who’s core expertise is selling cars, investing in creating a new web framework would not be to the advantage of getting sales quickly and is likely to add unnecessary expense. If your expertise is a new web technology that allows you to experience a car prior to driving (and closes more sales) that is where you should consider building your own technology.
When bootstrapping a company, the best decisions I have seen is to fit together as many pre-built components as you can, even if they are your presumed “core expertise”. Only when they begin to show that they are not flexible to your core expertise, should you consider replacing them with some newer, better, faster technology built by your own team. The time saved getting your product to market can help you evaluate whether you have a market for what you want, and you can later decide where your time is best spent in buliding for your customer.
In talking with other technologists and entreprenuers, the wisdom has generally been “fail fast”, so that you can limit investments down dead ends. You can summarize this as spend the money or take the extra space to try something rather then waiting to build it yourself. The time spent will be minimized and your can always pursue the next better idea if the current idea doesn’t work.
Time vs Money/Space is not a correct tradeoff, as they are not interchangeable. With Money and Space, you can always re-use it, loan it or find somebody who can get it for you cheaper. When it comes to Time, you are not getting it back.
One of the best things you can do for you career at any company and across all your companies is a keep a personal copy of every email where someone says “Good Job”, “Great work”, “Your absolutely right”, and “This team delivered the impossible”.
The purpose of keeping a separate folder for these types of email is two-fold:
- Every year when you have your annual review its great to add direct attributions that come from your colleagues, peers, and manager complimenting your work throughout the review period. It especially helps if you can counterweight your inevitable mistakes with praises from the CEO on your work on behalf of the company. ,
- Every once in a while you will have a difficult day for reasons you can control and many that you cannot. When those bad days happen, having a folder full of affirmations of all the spectacular things your have done before really helps. 
If you find you aren’t getting enough compliments and praise then you should either change your approach or change to a role where you can be successful.
 “Ruben, you rock!” — Jeff Bezos.
 “I can’t congratulate you enough or say enough about how I feel about this team. I worked with many good teams, but nothing like that, not even close. I love you all! Now let’s get back to work…. ” — Udi Manber
 “You gave me some really great advice last year. I’ve been saving the praise I’ve received for my annual review. While it’ll be great for that, it’s also just great to have for the occasional work-related rainy day. Thanks, Ruben ” -kim.