Sri, a junior studying psychology at The City College of New York, understood how challenging finding a great job can be. She had long dreamed about working in tech, but as a non-technical major, and with Silicon Valley 3,000 miles away, she mentally prepared herself for a tough job search.
Sri thought her problem would be getting access to employers. But she found it even more challenging to get the information and insights about companies, jobs and career paths that she needed to be successful. As a first generation college student, she didn’t have parents she could turn to for career advice. The idea that she might get an interview at places like Apple or Google, let alone a job, seemed nearly impossible.
How could a student like Sri get her foot in the door? Can psychology majors have successful careers in tech? How do the culture and entry-level experiences compare across different companies? There was information online from experienced employees at these companies, but Sri wanted to hear directly from other students, and she wanted information she could trust.
Introducing Reviews and Q&A to Handshake
Talking to many students, we found Sri’s experience was far from unique. We realized Handshake is uniquely positioned to help foster a learning environment across our community of 9 million students and young alumni from more than 500 schools of every size, type and location. So, for the past 9 months, our team has been working to develop ways to evolve the Handshake community and democratize information for more students like Sri.
We’re proud to announce the addition of Reviews and Q&A (Question & Answer) to Handshake. Reviews allow students to learn from the job and internship experiences of other students across the Handshake network. With Q&A, we’re creating a community for students to ask questions and get career advice from alumni and students with relevant experience. We’re excited by the potential of both initiatives to empower all students with a more informed career exploration and job search process.
Below, you can learn more about:
- Our 9-month process to develop Reviews and Q&A with deep input from students, Career Centers, employers, and experts on content and privacy law.
- The strong results from our pilot this spring with 171 partner schools — 51% more applications and 47% more engagement for students who read Reviews.
- How our content moderation policies and processes ensured high quality and professional content, without imposing a burden on schools.
- How employers responded enthusiastically to this new opportunity to engage with students.
- What’s next for Reviews and Q&A this fall — working to roll these powerful features out to all students across the Handshake network!
Evolving Handshake to Support Students
The idea for Reviews and Q&A stemmed from many conversations with students and career service staff. Last Fall, we set out on a six-month research process to explore new ideas that could help break down information barriers for all students. We spoke with dozens of students from different majors and backgrounds, and campuses big and small.
As with anything we build, it’s important to us that Handshake’s university partners are a core part of the process. We convened a series of day-long design workshops with more than 35 career services leaders from universities like Rutgers, Spelman, University of Illinois, and Stanford. Together, we discussed innovative ways to encourage more peer-to-peer learning across the Handshake community. This feedback was tremendously helpful in shaping Reviews and Q&A.
Pilot Program Results
We’ve been piloting these new features at 171 of our partner universities (some with Reviews, some with Q&A, and some with both). We’re excited to share the results and learnings from the pilot.
Reviews:
- We collected 16,000 reviews (10, 000 students chose to make their reviews public, 6,000 decided to make their reviews private).
- Students reviewed 6,000 employers, of all sizes, industries and geographies.
- Students who engaged with reviews were 51% more likely to submit an application. Students told us that reviews helped them find employers they wouldn’t have previously considered, and feel more confident about applying.
- Students who read reviews were also 47% more likely to return to Handshake over a two-week period. Students told us they came back more often because their first experience with reviews was so helpful.
Q&A:
- We collected hundreds of questions and answers from students and alumni, and piloted with 15,000 students
- Students who engaged with Q&A were 63% more likely to submit an application. Students told us that, like Reviews, reading thoughtful answers to relevant questions helped them feel more confident about applying to related jobs.
- Students who engaged with Q&A were 32% more likely to return to Handshake over a two-week period. Many students told us that they came back to Handshake specifically to dive into more Q&A on topics they were interested in.
- Alumni were eager to continue helping: “if any students want to get in touch, feel free to share my email!”
The pilot also helped us make some big improvements to our original plans, based off the data and feedback we received. For example, we heard from students that seeing a 1-to-5 star rating of a company wasn’t as helpful as more thoughtful, written reviews. So, we decided to eliminate ratings and focus instead on highlighting differentiating company attributes (e.g. “Inclusive Culture” or “Great Mentorship”). This feedback has helped us deliver a more engaging and useful experience for students.
Silvia, a senior at the University of Miami, found particular value from reading through reviews about Citibank, where she’ll be working this summer. “I loved getting the whole perspective of what other students liked, what could’ve been better, and the advice they’d share.I was pretty anxious about preparing for my internship. Getting the ‘inside scoop’ from previous Citi interns gave me actionable insights. I’m feeling a lot more confident. I only wish I had these Reviews when I was applying and interviewing!”
Silvia’s and Seo Young’s experiences are only some examples of how Handshake is democratizing information. With each new review or question answered, we get closer to our goal of ensuring every student has the insights necessary to be successful in their job search and career.
Content Moderation Process Proven Effective
Our pilot program also helped us test and improve our content and moderation policies at scale. These operations are important to ensure that Reviews and Q&A are helpful to students and also professional — no spam, profanity, personal criticism, discriminatory language, etc.
We began the process of developing these policies by consulting with privacy and employment counsel. We also examined best practices at platforms like Medium and Quora, which focus on delivering quality user-generated content. The input from our university partners in 1:1 calls and in our workshops helped ensure the moderation processes worked at scale and met the unique needs of universities.
Of the 10,000 public reviews submitted, 94% passed our content moderation process and were published. Submitted reviews were first passed through an automatic text analysis to check for profanity or discriminatory language. A team of Handshake moderators then read every review. Of the 6% that did not pass our process, the majority were empty reviews, where the student hadn’t written any qualitative feedback.
After publication, any user on Handshake (Career Center, Student or Employer) can flag a review, if they believe it violates our content guidelines. Only four reviews (0.04% of those published) were flagged post-publication for additional moderation. Most of these were reviews submitted to the wrong branch of a company, and were quickly rectified.
Our content guidelines and moderation process has been a key differentiator for Handshake Reviews compared to other online job review sites, ensuring we see high quality, constructive and professional content across the board. Importantly, since Handshake managed the moderation process end-to-end, there was no burden on our university partners .
Employers Find Value As Well
When we first proposed launching Reviews and Q&A, some university partners expressed concern that employers would object to being reviewed. Through the pilot, we’ve found that employers have been overwhelmingly excited about the ability to receive feedback and engage with students. While reviews cover a majority of Fortune 500 companies, it’s been especially rewarding to see how reviews have helped local employers to shine. They’ve helped increase the brand awareness of smaller employers that offer great opportunities, but can be often be overlooked by students.
Democratizing Information for All Students
With the success of this pilot, we are working to introduce Reviews and Q&A to all Handshake schools this Fall, so that all students can benefit from the collective insight of members across the Handshake network.
Imagine the impact we can make for the Sris at every school. Working together, we can ensure all students have equal access to the high quality information and insights they need to achieve successful outcomes and careers.