We are in the process of rolling out our next test to verify the default behaviors on new search are the correct default behaviors. You may recall that our earlier test indicated we should remove Soundex matches from the default experience. This test will help us determine if we should, by default, tighten search results even more.
The changes we will be testing include:
1) Whether Phonetic algorithm matches should be included by default on new search. For this test, we will be comparing how successful searchers are under the current default behavior (with Phonetic matches on) vs. Phonetic matches off (we'll still include "similar" matches).
2) Whether there should be some degree of first name matching. Currently the website does not require any matching on first name at all. This is why you can get results for Peter Blow when you search for Joe Blow. For this test we will compare successes from the current default behavior to one where the results must match exactly, or phonetically, or similar, or because of initials.
We will be analyzing performance of the different experiences across different user types and deciding upon the configuration that works best for the most users. (I think I have a pretty good idea about what configuration you all will be pulling for.)
As with our last test, this test reflects our commitment to taking a reasoned approach to significant changes to the user experience.