The optimal weight g

We begin by noting that for any training example $\Phi_j$ for which $s_T(d_j,q_j)=0$ and $s_B(d_j,q_j)=1$, the score computed by Equation 14 is $1-g$. In similar fashion, we may write down the score computed by Equation 14 for the three other possible combinations of $s_T(d_j,q_j)$ and $s_B(d_j,q_j)$; this is summarized in Figure 6.6 .

Figure 6.6: The four possible combinations of $s_T$ and $s_B$.

Let $n_{01r}$ (respectively, $n_{01n}$) denote the number of training examples for which $s_T(d_j,q_j)=0$ and $s_B(d_j,q_j)=1$ and the editorial judgment is Relevant (respectively, Non-relevant). Then the contribution to the total error in Equation 17 from training examples for which $s_T(d_j,q_j)=0$ and $s_B(d_j,q_j)=1$ is

$$[1-(1-g)]^2n_{01r} + [0-(1-g)]^2n_{01n}.$$ (18)

By writing in similar fashion the error contributions from training examples of the other three combinations of values for $s_T(d_j,q_j)$ and $s_B(d_j,q_j)$ (and extending the notation in the obvious manner), the total error corresponding to Equation 17 is

$$(n_{01r}+n_{10n})g^2+(n_{10r}+n_{01n})(1-g)^2 + n_{00r} + n_{11n}.$$ (19)

By differentiating Equation 19 with respect to $g$ and setting the result to zero, it follows that the optimal value of $g$ is

$$\frac{n_{10r}+n_{01n}}{n_{10r}+n_{10n}+n_{01r}+n_{01n}}.$$ (20)

Exercises.

• When using weighted zone scoring, is it necessary for all zones to use the same Boolean match function?

• In Example 6.1.1 above with weights $g_1=0.2, g_2=0.31$ and $g_3=0.49$, what are all the distinct score values a document may get?

• Rewrite the algorithm in Figure 6.4 to the case of more than two query terms.

• Write pseudocode for the function WeightedZone for the case of two postings lists in Figure 6.4 .

• Apply Equation 20 to the sample training set in Figure 6.5 to estimate the best value of $g$ for this sample.

• For the value of $g$ estimated in Exercise 6.1.3, compute the weighted zone score for each (query, document) example. How do these scores relate to the relevance judgments in Figure 6.5 (quantized to 0/1)?

• Why does the expression for $g$ in (20) not involve training examples in which $s_T(d_t,q_t)$ and $s_B(d_t,q_t)$ have the same value?