Render your AR app using Vulkan on Android NDK (C)

When the ArTextureUpdateMode is set to AR_TEXTURE_UPDATE_MODE_EXPOSE_HARDWARE_BUFFER, ARCore will provide an Android hardware buffer when ArSession_update() is called. This hardware buffer can be bound to a Vulkan VkImage.

View the sample application

Vulkan rendering support is demonstrated in the hello_ar_vulkan_c sample app.

Enable the hardware buffer output mode

The configured ArTextureUpdateMode determines how ARCore will update the camera texture. When it is set to AR_TEXTURE_UPDATE_MODE_EXPOSE_HARDWARE_BUFFER, ARCore will provide the camera image through a AHardwareBuffer.

Configure the session to use AR_TEXTURE_UPDATE_MODE_EXPOSE_HARDWARE_BUFFER:

ArConfig* ar_config = NULL;
ArConfig_create(ar_session, &ar_config);
ArConfig_setTextureUpdateMode(ar_session, ar_config,
                              AR_TEXTURE_UPDATE_MODE_EXPOSE_HARDWARE_BUFFER);
CHECK(ArSession_configure(ar_session, ar_config) == AR_SUCCESS);
ArConfig_destroy(ar_config);

Note: When using AR_TEXTURE_UPDATE_MODE_EXPOSE_HARDWARE_BUFFER on incompatible devices, ArSession_configure() will fail and return AR_ERROR_UNSUPPORTED_CONFIGURATION.

Obtain the hardware buffer

When AR_TEXTURE_UPDATE_MODE_EXPOSE_HARDWARE_BUFFER is enabled, use ArFrame_getHardwareBuffer() to get the hardware buffer:

void* native_hardware_buffer = NULL;
ArFrame_getHardwareBuffer(ar_session, ar_frame, &native_hardware_buffer);

if ((int64_t)native_hardware_buffer == 0) {
  // The hardware buffer isn't ready yet.
  return;
}

Note: This AHardwareBuffer is only guaranteed to be valid until the next call to ArSession_update(). If you want to use the hardware buffer beyond that, such as for rendering, you must call AHardwareBuffer_acquire() and then call AHardwareBuffer_release() after your rendering is complete.

Use the hardware buffer during Vulkan rendering

See vulkan_handler.cc for an example of how to render an AR application using Vulkan.

Supported devices

Vulkan rendering support is only available on Android API levels 27 and above. Additionally, the device must support the VK_ANDROID_external_memory_android_hardware_buffer extension.

Require Vulkan in your app's manifest

Google Play uses <uses-feature> declared in your app manifest to filter your app from devices that don't meet its hardware and software feature requirements. Devices using Vulkan 1.0 might not support the required extension, but devices compatible with Vulkan 1.1 must have the required extension starting in Android 10 (API level 29).