Areas of Interest

In the eyecode library, an area of interest (AOI) is a rectangle that has the following properties:

• A kind or layer
• A name
• A bounding box (x, y, width, height)

Multiple kinds of AOIs can be defined, but AOIs within a single kind (or layer) should not overlap. For example, AOIs with a “line” kind and a “token” kind could be defined for a program. Line AOIs should not overlap with each other, but they can freely overlap with token AOIs.

Creating AOIs

AOIs for lines and whitespace-separated tokens can be automatically identified from a black and white image of the code using the find_rectangles function.

eyecode.aoi.find_rectangles(screen_image, black_thresh=255, white_row_thresh=3, white_col_thresh=3, vert_kind='line', horz_kind='sub-line')

Scans a black and white code image for line and sub-line rectangles.

Parameters :

screen_image : PIL.Image

Image with code (will be converted to ‘L’ mode)

black_thresh : int, optional

Luminescence threshold for deciding a pixel is black (default: 255)

white_row_thresh : int, optional

Number of white rows before deciding a rectangle is done (default: 3)

vert_kind : str, optional

AOI kind to assign to all vertical rectangles (default: line)

horz_kind : str, optional

AOI kind to assign to all horizontal rectangles (default: sub-line)

Returns :

pandas DataFrame :

A dataframe with rectangle coordinates and sizes

See also

eyecode.plot.aoi.draw_rectangles

Visualize AOI rectangles

Examples

>>> from eyecode import aoi, data
>>> code_img = data.busjahn_2013.program_image("basketball")
>>> code_aois = aoi.find_rectangles(code_img)
>>> print code_aois[:3]
       kind           name    x   y  width  height
0      line         line 1  335  28    212      20
1  sub-line  line 1 part 1  335  28     53      20
2  sub-line  line 1 part 2  392  28     47      20

Hit Testing

Assigning fixations to AOIs is done using the hit_test function. As input, it takes dataframes with fixations and AOIs. The result is a copy of the fixations dataframe with additional columns for each AOI kind. The value of each AOI column is the hit AOI name (or NaN if no AOI was hit).

For example, hit testing fixations with AOIs whose kind was “line” and whose names were “line 1”, “line 2”, etc. would result in a dataframe with an “aoi_line” column. The value in this column would be “line 1” when the fixation hit line 1, “line 2” for line 2, and so on. If no line was hit, the value would be NaN (pandas default null value).

eyecode.aoi.hit_test(fixations, aois, offsets=None, hit_fun=<function hit_circle at 0x412c230>, hit_radius=20, **kwargs)

Hit tests fixations against AOI rectangles.

Parameters :

fixations : pandas DataFrame

A DataFrame with fixations to hit test (fix_x, fix_y)

aois : pandas DataFrame

A DataFrame with areas of interest (kind, name, x, y, width, height)

offsets : pandas DataFrame or None

A DataFrame with different fixations offsets to apply (name, x, y). If None, no offset is applied

hit_fun : callable

Hit testing function. See hit_point and hit_circle for examples

hit_radius : int

Fixation circle radius for hit_circle

Returns :

aoi_fixations : pandas DataFrame

A copy of the fixations DataFrame with additional columns for each offset and AOI kind

Notes

Requires the shapely library: http://toblerity.org/shapely

Examples

>>> from eyecode import aoi, data
>>> code_img = data.busjahn_2013.program_image("basketball")
>>> code_aois = aoi.find_rectangles(code_img)
>>> raw_fixes = data.busjahn_2013.raw_fixations()
>>> print raw_fixes[:5][["trial_id", "start_ms", "fix_x", "fix_y"]]
   trial_id  start_ms       fix_x       fix_y
0         8       250  423.437500  378.083344
1         8       567  324.711548   67.538460
2         8       867  415.625000   -3.750000
3         8      1284  444.852936  159.117645
4         8      2034  366.030792  133.842896
>>> aoi_fixes = aoi.hit_test(raw_fixes, sub_line_aois)
>>> aoi_cols = aoi.get_aoi_columns(aoi_fixes)
>>> print aoi_fixes[:5][["trial_id", "start_ms", "fix_x", "fix_y"] + aoi_cols]
   trial_id  start_ms       fix_x       fix_y aoi_line   aoi_sub-line
0         8       250  423.437500  378.083344   line 9  line 9 part 1
1         8       567  324.711548   67.538460      NaN            NaN
2         8       867  415.625000   -3.750000      NaN            NaN
3         8      1284  444.852936  159.117645   line 4  line 4 part 2
4         8      2034  366.030792  133.842896      NaN            NaN

Utility Methods

Below are a few utility functions for making AOI manipulation easier.

eyecode.aoi.envelope(aois, padding=0)

Returns a rectangle that envelopes the given AOI rectangles.

Parameters :

aois : pandas DataFrame

A dataframe with a row for each AOI (x, y, width, height)

Returns :

bbox : list of int

Bounding box around all aois (x, y, width, height)

eyecode.aoi.pad(aois, padding)

Pads the given AOIs.

Parameters :

aois : pandas DataFrame

A dataframe with a row for each AOI (x, y, width, height)

padding : int or list of int

Uniform padding (int) or top, right, bottom, left (list of int)

Returns :

padded_aois : pandas DataFrame

A copy of the input aois with padding applied

eyecode.aoi.add_bbox(aois, bbox, kind, name)

Adds a new AOI with the given bounding box, kind, and name.

Parameters :

aois : pandas DataFrame

A dataframe with a row for each AOI (x, y, width, height)

bbox : list of int

Bounding box of new AOI (x, y, width, height)

kind : str

New AOI kind

name : str

New AOI name

Returns :

more_aois : pandas DataFrame

A copy of the input aois with the new AOI appended

eyecode.aoi.get_aoi_columns(fixations)

Gets all columns in a dataframe that hold AOI names.

Parameters :

fixations : pandas DataFrame

A dataframe with a row for each fixation

Returns :

list of str :

Column names that correspond to AOI kinds