Social activity measurement by counting faces captured in first-person view lifelogging video

This paper proposes a method to measure the daily face-to-face social activity of a camera wearer by detecting faces captured in first-person view lifelogging videos. This study was inspired by pedometers used to estimate the amount of physical activity by counting the number of steps detected by accelerometers, which is effective for reflecting individual health and facilitating behavior change. We investigated whether we can estimate the amount of social activity by counting the number of faces captured in the first-person view videos like a pedometer. Our system counts not only the number of faces but also weighs in the numbers according to the size of the face (corresponding to a face's closeness) and the amount of time it was shown in the video. By doing so, we confirmed that we can measure the amount of social activity based on the quality of each interaction. For example, if we simply count the number of faces, we overestimate social activities while passing through a crowd of people. Our system, on the other hand, gives a higher score to a social actitivity even when speaking with a single person for a long time, which was also positively evaluated by experiment participants who viewed the lifelogging videos. Through evaluation experiments, many evaluators evaluated the social activity high when the camera wearer speaks. An interesting feature of the proposed system is that it can correctly evaluate such scenes higher as the camera wearer actively engages in conversations with others, even though the system does not measure the camera wearer's utterances. This is because the conversation partners tend to turn their faces towards to the camera wearer, and that increases the number of detected faces as a result. However, the present system fails to correctly estimate the depth of social activity compared to what the camera wearer recalls especially when the conversation partners are standing out of the camera's field of view. The paper briefly descibes how the results can be improved by widening the camera's field of view.