Our research explores a seamless interaction with smart devices, which we divide into three stages: (i) device recognition, (ii) user input recognition, and (iii) inferring the appropriate action to be carried out on the device (cf. Figure 1). We leverage wearable computers and combine them into one interaction system. This makes interactions ubiquitous in two ways: While smart devices are becoming increasingly ubiquitous, our wearable system will be ubiquitously available to the user, making it possible to interact everywhere, any time.