{"id":6886,"date":"2024-10-29T13:47:58","date_gmt":"2024-10-29T13:47:58","guid":{"rendered":"http:\/\/staging.diggibyte.com\/Diggibyte_57\/?p=1850"},"modified":"2025-06-03T10:06:42","modified_gmt":"2025-06-03T10:06:42","slug":"simplifying-data-governance-with-ucx","status":"publish","type":"post","link":"https:\/\/staging.diggibyte.com\/Diggibyte_57\/simplifying-data-governance-with-ucx\/","title":{"rendered":"Simplifying Data Governance with UCX: A Practical Guide to Installation and Configuration on Databricks\u00a0"},"content":{"rendered":"\r\n<p>In <em>today\u2019s data-driven world, effective data governance is paramount to ensure data quality, security, and compliance. Unity Catalog offers a robust solution for organizations seeking to streamline their data management practices. This comprehensive guide will walk you through the process of installing and configuring UCX on Databricks, empowering you to simplify data governance and unlock the full potential of your data, <\/em><strong><em>The companion for upgrading to Unity Catalog.<\/em><\/strong>\u00a0<\/p>\r\n\r\n\r\n\r\n<h4 class=\"wp-block-heading\"><strong>Requirements:<\/strong>\u00a0<\/h4>\r\n\r\n\r\n\r\n<ul class=\"wp-block-list\">\r\n<li>Python Version: 3.6 or higher\u00a0<\/li>\r\n\r\n\r\n\r\n<li>Databricks CLI: Installed and configured on your local machine\u00a0<\/li>\r\n\r\n\r\n\r\n<li>It is Mandatory to use Databricks Premium or Enterprise Workspace.\u00a0<\/li>\r\n\r\n\r\n\r\n<li><strong>Attach your Databricks Workspace to Metastore<\/strong>\u00a0<\/li>\r\n\r\n\r\n\r\n<li>Find More <a href=\"https:\/\/github.com\/databrickslabs\/ucx?tab=readme-ov-file#installation\" target=\"_blank\" rel=\"noreferrer noopener\">here<\/a>\u00a0<\/li>\r\n<\/ul>\r\n\r\n\r\n\r\n<h4 class=\"wp-block-heading\"><strong>Installing Databricks CLI (Windows)<\/strong>:<\/h4>\r\n\r\n\r\n\r\n<p>We will use `winget` to install the Databricks CLI. Make sure `winget` is available on your machine.\u00a0<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image aligncenter size-full is-resized\"><img decoding=\"async\" class=\"wp-image-1859\" style=\"width: 448px; height: auto;\" src=\"http:\/\/staging.diggibyte.com\/Diggibyte_57\/wp-content\/uploads\/2024\/10\/winget-install-databricks.jpg\" alt=\"\"><\/figure>\r\n\r\n\r\n\r\n<pre class=\"wp-block-preformatted\">winget search databricks <br \/> <br \/>winget install Databricks.DatabricksCLI <\/pre>\r\n\r\n\r\n\r\n<h4 class=\"wp-block-heading\"><strong>Installing Databricks CLI For macOS using\u202fHomebrew<\/strong>:<\/h4>\r\n\r\n\r\n\r\n<p>If you\u2019re a Mac user, installing tools via Homebrew is a no-brainer. It\u2019s a package manager that simplifies the installation of software on macOS, making your life easier. Here\u2019s a quick and painless guide to get the Databricks CLI set up using Homebrew.\u00a0<\/p>\r\n\r\n\r\n\r\n<h5 class=\"wp-block-heading\"><strong>Step 1: Check Homebrew Installation<\/strong>:<\/h5>\r\n\r\n\r\n\r\n<pre class=\"wp-block-preformatted\">brew -v\u00a0<\/pre>\r\n\r\n\r\n\r\n<h5 class=\"wp-block-heading\"><strong>Step 2: Tap Databricks Homebrew Repository<\/strong>:\u00a0<\/h5>\r\n\r\n\r\n\r\n<p>Next, you\u2019ll need to add the official Databricks Homebrew tap. A \u201ctap\u201d is essentially a repository for Homebrew packages that aren\u2019t included by default.\u00a0<\/p>\r\n\r\n\r\n\r\n<pre class=\"wp-block-preformatted\">brew tap Databricks\/tap\u00a0<\/pre>\r\n\r\n\r\n\r\n<h5 class=\"wp-block-heading\"><strong>Step 3: Install Databricks CLI<\/strong>:<\/h5>\r\n\r\n\r\n\r\n<p>With the tap added, you can now install the Databricks CLI:\u00a0<\/p>\r\n\r\n\r\n\r\n<pre class=\"wp-block-preformatted\">brew install Databricks\u00a0<\/pre>\r\n\r\n\r\n\r\n<p>This step pulls down the Databricks CLI and installs it for you. Homebrew handles the dependencies and takes care of the installation process.\u00a0<\/p>\r\n\r\n\r\n\r\n<h5 class=\"wp-block-heading\"><strong>Verify Installation<\/strong>:<\/h5>\r\n\r\n\r\n\r\n<p>After installation, verify that Databricks CLI is correctly set up by running the following command:\u00a0<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image aligncenter size-full is-resized\"><img decoding=\"async\" class=\"wp-image-1858\" style=\"width: 448px; height: auto;\" src=\"http:\/\/staging.diggibyte.com\/Diggibyte_57\/wp-content\/uploads\/2024\/10\/databricks-v.jpg\" alt=\"\">\r\n<figcaption class=\"wp-element-caption\">Databricks -v<\/figcaption>\r\n<\/figure>\r\n\r\n\r\n\r\n<h4 class=\"wp-block-heading\"><strong>Authenticate Databricks CLI<\/strong>:<\/h4>\r\n\r\n\r\n\r\n<p>Once the CLI is installed, authenticate it with your Databricks workspace by running:\u00a0<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image aligncenter size-large is-resized\"><img decoding=\"async\" class=\"wp-image-1857\" style=\"width: 583px; height: auto;\" src=\"http:\/\/staging.diggibyte.com\/Diggibyte_57\/wp-content\/uploads\/2024\/10\/databricks-auth-1024x67.jpg\" alt=\"\">\r\n<figcaption class=\"wp-element-caption\">Databricks auth login \u2013 host <WORKSPACE_HOST>\u00a0<\/figcaption>\r\n<\/figure>\r\n\r\n\r\n\r\n<p>Replace `<WORKSPACE_HOST>` with the actual URL of your Databricks workspace. You will be prompted to authenticate using your default browser to the Databricks authentication window.\u00a0<\/p>\r\n\r\n\r\n\r\n<h4 class=\"wp-block-heading\"><strong>Install\u202fUCX<\/strong>:<\/h4>\r\n\r\n\r\n\r\n<p>After successful authentication, you can now install UCX using the Databricks Labs extension:\u00a0<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image aligncenter size-full is-resized\"><img decoding=\"async\" class=\"wp-image-1856\" style=\"width: 520px; height: auto;\" src=\"http:\/\/staging.diggibyte.com\/Diggibyte_57\/wp-content\/uploads\/2024\/10\/databricks-labs-install-ucx.jpg\" alt=\"\">\r\n<figcaption class=\"wp-element-caption\">Databricks labs install ucx<\/figcaption>\r\n<\/figure>\r\n\r\n\r\n\r\n<h4 class=\"wp-block-heading\"><strong>UCX Configuration<\/strong>:<\/h4>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image aligncenter size-full is-resized\"><img decoding=\"async\" class=\"wp-image-1854\" style=\"width: 450px; height: auto;\" src=\"http:\/\/staging.diggibyte.com\/Diggibyte_57\/wp-content\/uploads\/2024\/10\/ucx-configure-1.jpg\" alt=\"\"><\/figure>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image aligncenter size-full is-resized\"><img decoding=\"async\" class=\"wp-image-1855\" style=\"width: 458px; height: auto;\" src=\"http:\/\/staging.diggibyte.com\/Diggibyte_57\/wp-content\/uploads\/2024\/10\/ucx-configure-2.jpg\" alt=\"\"><\/figure>\r\n\r\n\r\n\r\n<p>During the installation, you will be asked a series of questions about how to configure UCX. You can choose to keep the default settings for a basic setup. However, you can configure it based on your requirements<\/p>\r\n\r\n\r\n\r\n<h4 class=\"wp-block-heading\"><strong>Verify the Installation<\/strong>:<\/h4>\r\n\r\n\r\n\r\n<p>Once the UCX command Databricks labs install UCX has completed successfully, the installation can be verified with the following steps:\u00a0<\/p>\r\n\r\n\r\n\r\n<ol class=\"wp-block-list\" start=\"1\">\r\n<li>Go to the Databricks Catalog Explorer and check if a new schema for UCX is available in Hive Metastore with all empty tables.\u00a0<\/li>\r\n<\/ol>\r\n\r\n\r\n\r\n<ol class=\"wp-block-list\" start=\"2\">\r\n<li>Check that the UCX jobs are visible under Workflows.\u00a0<\/li>\r\n<\/ol>\r\n\r\n\r\n\r\n<h4 class=\"wp-block-heading\"><strong>Running Workflows<\/strong>:<\/h4>\r\n\r\n\r\n\r\n<p>Run the assessment. This will start the UCX clusters, crawl through the workspace, and display results in the UCX dashboards. In the case of external HMS, verify from the results that the assessment has analyzed the external HMS tables. This will generate the desired dashboards, providing you with insights and visualizations based on the data processed.\u00a0<\/p>\r\n\r\n\r\n\r\n<p>The <strong>UCX assessment workflow<\/strong> is intended to <strong>only run once<\/strong>; <strong>re-running it is not supported.<\/strong> If the inventory and findings for a workspace need to be updated then first reinstall UCX by <a href=\"https:\/\/github.com\/databrickslabs\/ucx?tab=readme-ov-file#uninstall-ucx\" target=\"_blank\" rel=\"noreferrer noopener\">uninstalling<\/a> and <a href=\"https:\/\/github.com\/databrickslabs\/ucx?tab=readme-ov-file#install-ucx\" target=\"_blank\" rel=\"noreferrer noopener\">installing<\/a> it again.\u00a0<\/p>\r\n\r\n\r\n\r\n<h4 class=\"wp-block-heading\"><strong>Workflows<\/strong>:<\/h4>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image aligncenter size-full is-resized\"><img decoding=\"async\" class=\"wp-image-1853\" style=\"width: 398px; height: auto;\" src=\"http:\/\/staging.diggibyte.com\/Diggibyte_57\/wp-content\/uploads\/2024\/10\/workflow.jpg\" alt=\"\"><\/figure>\r\n\r\n\r\n\r\n<h4 class=\"wp-block-heading\"><strong>Dashboards<\/strong>:<\/h4>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image aligncenter size-full is-resized\"><img decoding=\"async\" class=\"wp-image-1852\" style=\"width: 408px; height: auto;\" src=\"http:\/\/staging.diggibyte.com\/Diggibyte_57\/wp-content\/uploads\/2024\/10\/dashboard.jpg\" alt=\"\"><\/figure>\r\n\r\n\r\n\r\n<h4 class=\"wp-block-heading\"><strong>Assessment Overview Dashboard<\/strong>:<\/h4>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image aligncenter size-full is-resized\"><img decoding=\"async\" class=\"wp-image-1851\" style=\"width: 404px; height: auto;\" src=\"http:\/\/staging.diggibyte.com\/Diggibyte_57\/wp-content\/uploads\/2024\/10\/overview.jpg\" alt=\"\"><\/figure>\r\n\r\n\r\n\r\n<p><strong>Congratulations <\/strong><strong>\ud83c\udf89<\/strong><strong> on successfully installing Databricks Labs\u202fUCX!<\/strong>\u00a0<\/p>\r\n\r\n\r\n\r\n<p>By completing this installation, you\u2019ve laid the foundation for leveraging the powerful capabilities of UCX to enhance your data management and governance practices.\u00a0<\/p>\r\n\r\n\r\n\r\n<h4 class=\"wp-block-heading\"><strong>Next Steps<\/strong>:<\/h4>\r\n\r\n\r\n\r\n<ul class=\"wp-block-list\">\r\n<li><strong>Explore UCX Features:<\/strong> Delve deeper into UCX\u2019s functionalities to discover how it can address your specific data management challenges.\u00a0<\/li>\r\n<\/ul>\r\n\r\n\r\n\r\n<ul class=\"wp-block-list\">\r\n<li><strong>Implement Governance Policies:<\/strong> Define and enforce data governance policies using UCX\u2019s capabilities.\u00a0<\/li>\r\n<\/ul>\r\n\r\n\r\n\r\n<ul class=\"wp-block-list\">\r\n<li><strong>Monitor and Optimize:<\/strong> Continuously monitor UCX performance and usage to identify opportunities for optimization.\u00a0<\/li>\r\n<\/ul>\r\n\r\n\r\n\r\n<h4 class=\"wp-block-heading\"><strong>Conclusion<\/strong>:<\/h4>\r\n\r\n\r\n\r\n<p>You can find the GitHub page of the official UCX <a href=\"https:\/\/github.com\/databrickslabs\/ucx\" target=\"_blank\" rel=\"noreferrer noopener\">here<\/a>\u00a0<\/p>\r\n\r\n\r\n\r\n<p>If you have trouble logging in to UCX, check the Databricks documentation for help. <a href=\"https:\/\/github.com\/databrickslabs\/ucx?tab=readme-ov-file#authentication-issues\" target=\"_blank\" rel=\"noreferrer noopener\">here<\/a>\u00a0<\/p>\r\n\r\n\r\n\r\n<p id=\"3837\">For More Details, <strong>Diggibyte Technologies Pvt Ltd<\/strong> has all the experts you need. Contact us Today to embed intelligence into your organization.<\/p>\r\n\r\n\r\n\r\n<blockquote class=\"wp-block-quote has-text-align-right is-layout-flow wp-block-quote-is-layout-flow\">\r\n<p>Author: Basheer Ahmed<\/p>\r\n<\/blockquote>\r\n\r\n\r\n\r\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\r\n<div class=\"wp-block-button is-style-primary\"><a class=\"wp-block-button__link wp-element-button\" href=\"http:\/\/staging.diggibyte.com\/Diggibyte_57\/contact.html\" target=\"_blank\" rel=\"noreferrer noopener\">Contact Us<\/a><\/div>\r\n<\/div>\r\n\r\n\r\n\r\n<p>\u00a0<\/p>\r\n","protected":false},"excerpt":{"rendered":"<p>In today\u2019s data-driven world, effective data governance is paramount to ensure data quality, security, and compliance. Unity Catalog offers a robust solution for organizations seeking to streamline their data management practices. This comprehensive guide will walk you through the process of installing and configuring UCX on Databricks, empowering you to simplify data governance and unlock [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":7144,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[126],"tags":[26,27,28,30,31,90,83,114,115,116,52],"class_list":["post-6886","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-databricks","tag-analytics","tag-bigdata","tag-data","tag-dataanalysis","tag-dataanalytics","tag-datamanagement","tag-datavisualization","tag-fabric","tag-microsoftfabric","tag-microsoftpowerbi","tag-powerbi"],"_links":{"self":[{"href":"https:\/\/staging.diggibyte.com\/Diggibyte_57\/wp-json\/wp\/v2\/posts\/6886","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/staging.diggibyte.com\/Diggibyte_57\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/staging.diggibyte.com\/Diggibyte_57\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/staging.diggibyte.com\/Diggibyte_57\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/staging.diggibyte.com\/Diggibyte_57\/wp-json\/wp\/v2\/comments?post=6886"}],"version-history":[{"count":2,"href":"https:\/\/staging.diggibyte.com\/Diggibyte_57\/wp-json\/wp\/v2\/posts\/6886\/revisions"}],"predecessor-version":[{"id":8194,"href":"https:\/\/staging.diggibyte.com\/Diggibyte_57\/wp-json\/wp\/v2\/posts\/6886\/revisions\/8194"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/staging.diggibyte.com\/Diggibyte_57\/wp-json\/wp\/v2\/media\/7144"}],"wp:attachment":[{"href":"https:\/\/staging.diggibyte.com\/Diggibyte_57\/wp-json\/wp\/v2\/media?parent=6886"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/staging.diggibyte.com\/Diggibyte_57\/wp-json\/wp\/v2\/categories?post=6886"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/staging.diggibyte.com\/Diggibyte_57\/wp-json\/wp\/v2\/tags?post=6886"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}