Software Supreme helped improve speed of inferencing on the edge by 180%
A client doing R&D for ADAS had developed a set of neural networks in TensorFlow and PyTorch that they wanted to deploy efficiently into a C++ environment for inferencing on the edge.
Software Supreme was tasked with the integration of two networks – using TensorFlow and LibTorch’s C++ bindings. This solution worked, however both frameworks were fighting over the GPU resources, which led to poor device utilization, so we decided to switch to native TensorRT. To achieve that we implement a few custom layers as TensorRT plugins.. With TensorRT we had full control over the NN execution. We were able to address the initial bottleneck and further increase performance through the custom GPU accelerated layers.
Both networks were integrated and performance exceeded the target specs. One of the networks’ performance increased from 50fps up to 140fps. This allowed both networks to run in parallel each frame, as opposed to running sequentially.
Privacy & Cookies Policy
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.