Eye gaze is not coded by cardinal mechanisms alone

Gaze is an important social cue in regulating human and non-human interactions. In this study, we employed an adaptation paradigm to examine the mechanisms underlying the perception of another's gaze. Previous research has shown that the interleaved presentation of leftwards and rightwards gazing adaptor stimuli results in observers judging a wider range of gaze deviations as being direct. We applied a similar paradigm to examine how human observers encode oblique (e.g. upwards and to the left) directions of gaze. We presented observers with interleaved gaze adaptors and examined whether adaptation differed between congruent (adaptor and test along same axis) and incongruent conditions. We find greater adaptation in congruent conditions along cardinal (horizontal and vertical) and non-cardinal (oblique) directions suggesting gaze is not coded alone by cardinal mechanisms. Our results suggest that the functional aspects of gaze processing might parallel that of basic visual features such as orientation.


Introduction
Accurately perceiving the direction of another person's eye gaze plays an important social function [1], with evidence linking abnormal gaze behaviour to certain clinical populations (e.g. autism and schizophrenia). Gaze direction communicates information about the mental state, emotion and interest of another individual which can be used to understand the environment and the likely thoughts and future behaviour of that individual (for a review, see [2]). For example, direct gaze can communicate friendliness [3] or threat [2] and averted gaze can communicate avoidance [4] or interest in a particular location in the environment [5].
Most studies, however, have investigated the perception of gaze along a single cardinal axis, the horizontal [6][7][8][9]. Much less attention has been devoted in understanding coding mechanisms for vertical and non-cardinal (diagonal) gaze directions. Seminal single cell experiments by Perrett et al. [10] revealed that different cells respond to direct gaze and averted gaze in vertical and horizontal directions, respectively, suggesting the existance of channels coding gaze directed towards the left, right, up, down and straight-ahead. Hypothetically, these cardinal dimensions (horizontal and vertical) would be sufficient to encode all eye directions. The existence of more than two (cardinal) channels could provide us with greater sensitivity to detect small deviations in another's gaze, however, it remains an open question as to whether these non-cardinal channels exist.
As far as we are aware, only one study has explored gaze processing where eyes deviate in diagonal directions [11]. Gaze judgements were up to 48 per cent less accurate for gaze in diagonal directions relative to cardinal directions, and there was a tendency to bias diagonal gaze directions towards the cardinal axes [11]. However, it must be noted that these authors used a triadic paradigm involving three entities; a 'sender'(1) who looked at an object of interest (2) and a 'receiver'(3) who had to judge which object the sender was looking at. This is contrasted to much of the research on gaze that uses a dyadic paradigm where there are two entities; a 'sender' (1), and a receiver (2) who judges the direction of gaze of the 'sender' [6,7,10,12,13].
Preferential processing of cardinal directions has also been observed in studies of basic visual attributes, including motion [14,15] and orientation. For example, judgements of the orientation of lines is more accurate near the cardinal axes when compared with the diagonal axes [16]; and when oriented lines are presented under uncertainty, there is a tendency to judge them as tilted towards cardinal directions [17,18].
Thus, it appears there may be something unique about cardinal dimensions in both basic vision and gaze processing (gaze sensitive channels which are tuned along cardinal directions; [11]). Whether in the case of gaze, this reflects an absence of channels coding non-cardinal dimensions remains to be established, however.
Adaptation is a tool frequently used in vision research to examine underlying neural mechanisms (e.g. in colour processing; [19]). Adaptation causes a loss in responsiveness of the mechanisms which code the adapting stimulus [7,20]. A number of studies have examined perceptual changes after gaze adaptation. For example, adapting to interleaved leftwards and rightwards stimuli led to an increase in the range of gaze directions either side of direct gaze categorized as direct (the cone of direct gaze; [9]): i.e. small leftwards and rightwards deviations were more likely judged as being direct following adaptation [6].These effects persisted despite changes in the size of the test face, suggesting that adaptation is not occurring within low-level visual mechanisms that respond to contrast, but rather reflects the adaptation of high-level gaze-processing mechanisms [6]. However, highlevel adaptation of face-processing mechanisms has been found to follow a similar time course of build-up and decay as low-level adaptation, suggesting that analogous neural mechanisms might be involved in high-and low-level adaptation [21,22].
In this paper, we used a similar adaptation paradigm to investigate whether only two cardinal mechanisms (horizontal and vertical) exist to encode all gaze deviations. Stimuli were faces with eyes deviated along the horizontal, vertical and two non-cardinal (diagonal) axes: RU/LD (right-up/leftdown) and LU/RD (left-up/right-down). Participants adapted to eyes deviated 158 in one direction interleaved with eyes deviated 158 in the opposite direction along the same axis.
After adaptation, participants were tested with stimuli along the same axis (congruent trials) or the perpendicular axis (incongruent trials). We investigated the presence of a noncardinal gaze mechanism by comparing the cone of direct gaze on congruent and incongruent conditions. The reasoning is as follows; after adaptation to gaze on one non-cardinal axis, both cardinal axis mechanisms should be less responsive since adapting along the RU/LD axis adapts left, right, up and down directions. Similarly, adaptation along the LU/RD axis also results in (equal) adaptation of the two cardinal mechanisms. Therefore, if mechanisms exist along cardinal axes only, there should be no difference in the width of the cone of direct gaze between congruent and incongruent conditions along the non-cardinal axes.

Material and methods (a) Apparatus and stimuli
A Dell XPS computer running MATLAB (MathWorks Ltd) was used for stimulus generation, experiment control and recording subjects' responses. The programs controlling the experiment incorporated elements of the PsychToolbox [23]. Stimuli were displayed on a Sony Trinitron 20SE monitor (1024 Â 768 pixels, refresh rate: 75 Hz) driven by the computer's built-in NVIDIA GeForce GTS 240 graphics card. The display was calibrated using a photometer and linearized using look-up tables in software. At the viewing distance of 57 cm, one pixel subtended 2.2 arcmin. Two authors and six naive volunteer subjects (four females in total) aged between 23 and 57 years participated in the study. All participants had normal or corrected to normal vision. All experiments adhered to the Declaration of Helsinki guidelines.
Stimuli were grey scale synthetic faces (four males and four females) created with DAZ software (http://www.daz3d.com/). The hair was cropped and the face was presented within a circular aperture in the middle of the monitor (see figure 1 for sample face). The stimuli subtended 15.18 Â 11.28 and were viewed in a dimly lit room. The original eyes in the faces were replaced using GIMP software by greyscale eye stimuli created using MATLAB. The deviation of each eye was then independently controlled using MATLAB procedures that gave us precision down to the nearest pixel for eye rotation along any axis.
(b) Procedure: adaptation As discussed, adaptation to interleaved stimuli of opposite sign causes a widening in the cone of direct gaze [6]. We sought to determine (i) whether adaptation effects were measurable along the different axes of gaze and (ii) whether adaptation resulted in a wider cone of direct gaze following congruent adaptation compared with incongruent adaptation. Cones of direct gaze were measured in the following order: -Pre-adaptation baseline. The baseline cone of direct gaze along each axis was measured in two separate runs, once using female probe faces and once using male probe faces. Stimuli were presented using a method of constant stimuli with nine different directions of gaze selected from the set: (2128, 268, 238, 218, 08, 18, 38, 68, 128). Each stimulus was presented for 400 ms followed by a grey screen that lasted 600 ms during which no response was recorded. The next trial was only initiated after a response was made following the 600 ms wait period. Timings for the measurement of the pre-adaptation baselines were reduced compared with the study of Calder et al. [6] in order to minimize the overall duration of the full experiment. However, we ensured that the inter-stimulus interval (minimum of 600 ms) was long enough to avoid any potential motion or after-image cues. Each direction of gaze was sampled 12 times per run. -Adaptation. This stage involved adapting and testing to congruent and incongruent stimuli for each of the four axes. Between different adaptation conditions, observers took a break of at least 2 h to prevent any carryover adaptation effects. The adaptation was in figure 1. In part figure 1a, the subjects adapted to eyes deviated at þ158 for 4000 ms (in one direction along the axis tested) followed by a grey screen for 200 ms, then eyes at 2158 (the opposite direction along the axis tested) for 4000 ms. This was repeated for each of the four faces (presented at random) for a period of 1 min. This was followed immediately by a top-up adaptation and probe (figure 1b). In figure 1b, four faces (two at þ158 and two at 258) were presented for 1300 ms each, followed by a blank screen for 200 ms and then the probe stimulus presented for 500 ms. The probe stimuli were 33 per cent larger than, and rspb.royalsocietypublishing.org Proc R Soc B 280: 20131049 opposite gender to the adapting stimuli to increase their saliency and to minimize the adaptation of low-level mechanisms [6]. There was a 200 ms grey screen wait period following the observer's response. Each probe deviation angle was sampled 12 times in figure 1b (four faces Â three presentations each). Congruent and incongruent adaptation conditions were run twice separately (once adapting with female faces and adapting with male faces) in a random counterbalanced order across observers.
-Post-adaptation baseline. The baseline cones of direct gaze were measured at least 2 h after adaptation, and gender tested was presented in a counterbalanced order to the pre-adaptation sequence.
(c) Procedure: measuring the cone of direct gaze The observers' task in each stage was always the same; to indicate whether the direction of gaze in the probe face was direct or averted using number keys. Stimuli were presented along four axes and different number keys to record the observer's response were used accordingly: for the horizontal axis response keys were '4' (left), '5' (direct) and '6' (right), for the vertical axis response keys were '2' (up), '5' and '8' (down), for the LU/RD axis response keys were '1' (left-up), '5' and '9' (right-down) and for the RU/LD axis response keys were '3' (right-up), '5' and '7' (left-down). These different axes correspond to stimuli whose deviations were along the horizontal, vertical, oblique at þ458 (RU/LD) or oblique at 2458 (LU/RD). Each observer's data within the same condition and axis were compiled across runs and logistic functions were fitted to the proportion of 'left' and 'right' responses (for the horizontal axis). A function for 'direct' responses was calculated by subtracting the sum of the 'left' and 'right' responses from one. These three functions were fitted as an ensemble using the Nelder-Mead simplex method [24] implemented via Matlab's fminsearch function to minimize residual variance. The separation between the crossover points of the 'direct' and the 'left', and 'direct' and 'right' responses, respectively, is taken as the cone of direct gaze. A similar procedure was employed along the three other axes, plotting the proportion of RU/LD axis; LU/RD axis or 'down' and 'up' (vertical axis) responses.     Figure 2a plots the cones of direct gaze for one naive observer (O6) along the vertical axis after congruent adaptation (solid curve), incongruent adaptation (dashed curve) and in the average baseline ( pre-and post-adaptation baselines: grey curve). This observer shows selective effects of adaptation: the cone of direct gaze in the congruent condition is wider than baseline, whereas incongruent adaptation only moderately increased the cone of direct gaze. Figure 2b plots the cone of direct gaze after congruent adaptation (divided by baseline) against the cone of direct gaze after incongruent adaptation (divided by baseline) for all observers. Data points above the line of equality (dotted line) represent conditions where congruent adaptation resulted in a larger cone of direct gaze than incongruent adaptation. All the data for the cardinal axes fall above this line. Using a 2 Â 2 repeated measures ANOVA, we find that the cone of direct gaze did not vary according to adapting axis (averaged over probe axis conditions), F 1,7 , 0.01, p . 0.9, d , 0.01, or according to probe axis (averaged over adapting axis conditions) F 1,7 ¼ 2.17, p . 0.15, d ¼ 0.52. Congruent adaptation (adapt/probe horizontal or adapt/probe vertical) resulted in a significantly wider cone of direct gaze than incongruent adaptation (adapt horizontal/probe vertical or adapt vertical/probe horizontal), on average, F 1,7 ¼ 42.26, p , 0.01, d ¼ 2.30. Tests of simple effects revealed that the cone of direct gaze was significantly wider after congruent adaptation relative to incongruent adaptation when the probe was horizontal,   Table 2 reports the cone of direct gaze along non-cardinal axes measured in four conditions. Figure 3a plots data for one of the authors (O2) on the LU/RD probe axis, who showed a larger cone of direct gaze for congruent adaptation relative to incongruent adaptation. Using a 2 Â 2 repeated measures ANOVA, we find that the cone of direct gaze did not significantly differ according to adapting axis (LU/RD; RU/LD, averaged over probe axis conditions), F 1,7 ¼ 0.67 p . 0.4, d ¼ 0.29, or according to probe axis (LU/RD; RU/LD, averaged over adapting axis conditions) F 1,7 ¼ 0.66, p . 0.4, d ¼ 0.29. Congruent adaptation (adapt/probe (LU/RD) or adapt/probe (RU/ LD)) resulted in a significantly larger cone of direct gaze than incongruent adaptation (adapt (LU/RD)/probe (RU/ LD) or adapt (RU/LD)/probe (LU/RD)), on average, F 1,7 ¼ 24.11, p , 0.01, d ¼ 1.73. This can be visualized in figure 3b, where all but two data points for the non-cardinal axes fall above the line of equality (dotted line). Tests of simple effects revealed that the cone of direct gaze was significantly wider after congruent adaptation relative to incongruent adaptation when the probe was LU/RD, F 1,7 ¼ 9.14, p , 0.02, d ¼ 1.07, but narrowly missed significance when the probe stimulus was RU/LD, F 1,7 ¼ 4.32, p ¼ 0.076, d ¼ 0.73.

(c) Adaptation relative to baseline
To determine whether adaptation changed the size of the cone of direct gaze, we compared average baselines with adaptation data in a 4 Â 3 repeated measures ANOVA examining the effects of axis of the probe (horizontal, vertical, LU/RD and RU/LD) and adaptation condition (congruent cone width after incongruent adaptation (relative to baseline) cone width after congruent adaptation (relative to baseline)

Discussion
We find that adaptation is greater in congruent conditions compared with incongruent conditions for stimuli presented on cardinal and non-cardinal axes. We also report that congruent, but not incongruent, adaptation results in a significantly wider cone of direct gaze relative to baseline. Finally, in the absence of adaptation (baseline) the cones of direct gaze on the two non-cardinal axes and the vertical axes were similar in size and all significantly wider than the cone measured on the horizontal axis.
Our finding that adaptation along non-cardinal axes depends on adaptor-test congruence supports the existence of at least one non-cardinal mechanism that codes gaze. This is because adapting along either non-cardinal axis should have the same effect on cardinal mechanisms, such that if gaze is coded in terms of cardinal mechanisms alone, congruent and incongruent adaptation along non-cardinal axes would result in cones of direct gaze of a similar width. Instead our data indicate the existence of at least one channel tuned to gaze along an oblique axis. It is important to note that we do not refute that cardinal axes may play a unique role in gaze (as suggested by [11]), simply that gaze is not coded exclusively by these cardinal mechanisms.
As mentioned previously, all gaze directions could in principle be sufficiently be coded by two cardinal mechanisms. However, the results here indicate there must be at least one non-cardinal mechanism coding for gaze. There are many possible configurations for processing gaze directions with the presence of a gaze mechanism operating along a noncardinal axis. For example, there may be one cardinal and one non-cardinal mechanism, or a configuration involving three or more mechanisms. Presumably, a greater number of mechanisms would lead to an enhanced ability to resolve gaze directions accurately.
FMRI adaptation in humans [26] and previous electrophysiological recordings in monkeys [10] have revealed specific neural mechanisms that respond to different directions of gaze along the vertical and horizontal axes (in heads that were either forward facing or rotated). Yet, these experiments have not explored the specific angular tuning of such neurons. For example, a neuron responding to a 'left averted' gaze might actually have a preferred direction of 'left and slightly up'. More detailed physiological experimentation is needed to determine the relative prevalence and properties of cardinal and non-cardinal gaze sensitive cells. It is possible that the distribution of gaze neurons may mirror that seen in the coding of orientation, where there is a continuum of mechanisms tuned to different orientations, with a greater prevalence of mechanisms that code in cardinal directions [27 -29]. Indeed, there is already evidence that cells tuned to head orientation are more likely to exhibit a preference for cardinal head orientations (full face, back of head and left and right profiles) than intermediate orientations [30]. As far as we are aware, however, this has not been addressed for gaze direction.
An analysis of the baseline data reveals that a similar range of eye gaze deviations are categorized as direct along the vertical and two non-cardinal axes (oriented at þ458 and 2458 to the horizontal), while a more narrow range of eye gaze deviations are categorized as direct along the horizontal ( figure 4). This is consistent with Vida & Maurer [13], who report narrower cone of direct gaze on the horizontal axis relative to the vertical axis. Additionally, our estimates of the cone of direct gaze along the horizontal (and vertical) axis, 6.158 (and 7.338), are similar to earlier estimates, 5.498 (and 6.968, [13]) and 5.68 [31].
Specific gaze directions can communicate valuable social information. For example, direct gaze can indicate approach emotions (in the subject of interest) such as anger and joy, whereas averted gaze can indicate avoidance emotions such as sadness and fear [32]. Looking up and slightly averted to one side can indicate the subject is thinking [33], and averted gaze can also elicit mistrust in the observer [34]. Our results may bear on certain clinical populations who display an abnormal ability to represent and/or interpret the social meaning of specific gaze directions, such as people with autism [35][36][37], schizophrenia [38], social phobia [39], Turner's syndrome [40] and William's syndrome [41]. For example, school-aged children with autism are reported to perform more poorly than typical children when asked to make fine-grained judgements on gaze direction [35] or to identify the target of another's gaze [42]. Since additional gaze mechanisms could increase the ability to make accurate judgements of gaze direction, the reduced ability to make accurate judgements of gaze direction in schoolaged children with autism might be accounted for by the absence of one or more non-cardinal mechanisms for gaze.
People with autism have also been shown to lack normal adaptive mechanisms coding for social stimuli such as facial identity [43] and gaze [37]. It has been suggested that these results are consistent with a deficit in incorporating prior knowledge into the processing of such social stimuli [44]. It has recently been shown that normal adult observers have a prior expectation that another's gaze is directed towards them [45]. However, the findings of Pellicano et al. [37] and Mareschal et al. [45] are all based on stimuli in which gaze varied along the horizontal. Future work using non-cardinal stimuli such as those employed here would allow a fuller characterization of the normally functioning gaze-processing system and the deficits associated with autism spectrum disorder. Table 3. Mean cone of direct gaze for baselines measured before (pre) and after (post) adaptation. Note, H, V, LU/RD, RU/LD stands for the horizontal, vertical, left and up to right and down, and right and up to left and down axes, respectively. Means are in degrees of visual angle, standard deviation of the mean shown in brackets.