Reinforcement learning algorithms have proven to be capable of solving complicated robotics tasks in an end-to-end fashion without any need for hand-crafted features or policies. Especially in the context of robotics, in which the cost of real-world data is usually extremely high, Reinforcement Learning solutions achieving high sample efficiency are needed. In this paper, we propose a framework combining the learning of a low-dimensional state representation, from high-dimensional observations coming from the robot's raw sensory readings, with the learning of the optimal policy, given the learned state representation. We evaluate our framework in the context of mobile robot navigation in the case of continuous state and action spaces. Moreover, we study the problem of transferring what learned in the simulated virtual environment to the real robot without further retraining using real-world data in the presence of visual and depth distractors, such as lighting changes and moving obstacles. A video of our experiments can be found at: https://youtu.be/rUdGPKr2Wuo.

Indoor occupancy prediction is a prerequisite for the management of energy consumption, security, health, and other systems in smart buildings. Previous studies have shown that buildings that automatize their heating, lighting, air conditioning, and ventilation systems through considering the occupancy and activity information might reduce energy consumption by more than 50%. However, it is difficult to use high-resolution sensors and cameras for occupancy prediction due to privacy concerns. In this paper, we propose a novel solution for predicting occupancy using multiple low-cost and low-resolution heat sensors. We suggest two different methods for fusing and processing the data captured from multiple heat sensors and we use a Convolutional Neural Network for predicting occupancy. We conduct experiments to assess both the performance of the proposed solutions and analyze the impact of sensor field view overlaps on the prediction results. In summary, our experimental results show that the implemented solutions show high occupancy prediction accuracy and real-time processing capabilities.

Reinforcement Learning has been able to solve many complicated robotics tasks without any need for feature engineering in an end-to-end fashion. However, learning the optimal policy directly from the sensory inputs, i.e the observations, often requires processing and storage of a huge amount of data. In the context of robotics, the cost of data from real robotics hardware is usually very high, thus solutions that achieve high sample-efficiency are needed. We propose a method that aims at learning a mapping from the observations into a lower-dimensional state space. This mapping is learned with unsupervised learning using loss functions shaped to incorporate prior knowledge of the environment and the task. Using the samples from the state space, the optimal policy is quickly and efficiently learned. We test the method on several mobile robot navigation tasks in a simulation environment and also on a real robot. A video of our experiments can be found at: https://youtu.be/dgWxmfSv95U.

Collaborative swarms of robots/UAVs constitute a promising solution for precision agriculture and for automatizing agricultural processes. Since agricultural fields have complex topologies and different constraints, the problem of optimized path routing of these swarms is important to be tackled. Hence, this paper deals with the problem of optimizing path routing for a swarm of ground robots and UAVs in different popular topologies of agricultural fields. Four algorithms (Nearest Neighbour based on K-means clustering, Christofides, Ant Colony Optimisation and Bellman-Held-Karp) are applied on various farm types commonly found around Europe. The results indicate that the problem of path planning and the corresponding algorithm to use, are sensitive to the field topology and to the number of agents in the swarm.

Solving the challenge of occupancy prediction is crucial in order to design efficient and sustainable office spaces and automate lighting, heating, and air circulation in these facilities. In office spaces where large areas need to be observed, multiple sensors must be used for full coverage. In these cases, it is normally important to keep the costs low, but also to make sure that the privacy of the people who use such environments are preserved. Low-cost and low-resolution heat (thermal) sensors can be very useful to build solutions that address these concerns. However, they are extremely sensitive to noise artifacts which might be caused by heat prints of the people who left the space or by other objects, which are either using electricity or exposed to sunlight. There are some earlier solutions for occupancy prediction that employ low-resolution heat sensors; however, they have not addressed nor compensated for such heat artifacts. Therefore, in this paper, we presented a low-cost and low-energy consuming smart space implementation to predict the number of people in the environment based on whether their activity is static or dynamic in time. We used a low-resolution (8×8) and non-intrusive heat sensor to collect data from an actual meeting room. We proposed two novel workflows to predict the occupancy; one that is based on computer vision and one based on machine learning. Besides comparing the advantages and disadvantages of these different workflows, we used several state-of-the-art explainability methods in order to provide a detailed analysis of the algorithm parameters and how the image properties influence the resulting performance. Furthermore, we analyzed noise resources that affect the heat sensor data. The experiments show that the feature classification based method gives high accuracy when the data are clean from noise artifacts. However, when there are noise artifacts, the computer vision based method can compensate for those artifacts providing robust results. Because the computer vision based method requires an empty room recording, the feature classification based method should be chosen either when there is no expectancy of seeing noise artifacts in the data or when there is no empty recording available. We hope that our analysis brings light into understanding how to handle very low-resolution heat images in these environments. The presented workflows could be used in various domains and applications other than smart offices, where occupancy prediction is essential, e.g., for elderly care.

In this research, we investigate the use of Reinforcement Learning (RL) for an effective and robust solution for exploring unknown and indoor environments and reconstructing their maps. We benefit from a Simultaneous Localization and Mapping (SLAM) algorithm for real-time robot localization and mapping. Three different reward functions are compared and tested in different environments with growing complexity. The performances of the three different RL-based path planners are assessed not only on the training environments, but also on an a priori unseen environment to test the generalization properties of the policies. The results indicate that RL-based planners trained to maximize the coverage of the map are able to consistently explore and construct the maps of different indoor environments.