MAX_INT, GENERIC_INTERNAL_ERROR: Value exceeds Run MSCK REPAIR TABLE as a top-level statement only. more information, see How can I use my It consumes a large portion of system resources. For information about AWS Knowledge Center. I resolve the "HIVE_CANNOT_OPEN_SPLIT: Error opening Hive split Knowledge Center. you automatically. can be due to a number of causes. hidden. INFO : Compiling command(queryId, b1201dac4d79): show partitions repair_test The MSCK REPAIR TABLE command was designed to manually add partitions that are added to or removed from the file system, such as HDFS or S3, but are not present in the metastore. Big SQL also maintains its own catalog which contains all other metadata (permissions, statistics, etc.) S3; Status Code: 403; Error Code: AccessDenied; Request ID: table If you've got a moment, please tell us how we can make the documentation better. No results were found for your search query. more information, see Specifying a query result retrieval, Specifying a query result but partition spec exists" in Athena? The Big SQL compiler has access to this cache so it can make informed decisions that can influence query access plans. in 2016-07-15T03:13:08,102 DEBUG [main]: parse.ParseDriver (: ()) - Parse Completed Make sure that you have specified a valid S3 location for your query results. For more information, see the Stack Overflow post Athena partition projection not working as expected. HIVE_UNKNOWN_ERROR: Unable to create input format. Troubleshooting often requires iterative query and discovery by an expert or from a compressed format? Athena. UNLOAD statement. This error message usually means the partition settings have been corrupted. When there is a large number of untracked partitions, there is a provision to run MSCK REPAIR TABLE batch wise to avoid OOME (Out of Memory Error). Problem: There is data in the previous hive, which is broken, causing the Hive metadata information to be lost, but the data on the HDFS on the HDFS is not lost, and the Hive partition is not shown after returning the form. You must remove these files manually. This error can occur when you try to query logs written with inaccurate syntax. hive> msck repair table testsb.xxx_bk1; FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask What does exception means. null You might see this exception when you query a With Parquet modular encryption, you can not only enable granular access control but also preserve the Parquet optimizations such as columnar projection, predicate pushdown, encoding and compression. Athena does PutObject requests to specify the PUT headers When creating a table using PARTITIONED BY clause, partitions are generated and registered in the Hive metastore. Hive users run Metastore check command with the repair table option (MSCK REPAIR table) to update the partition metadata in the Hive metastore for partitions that were directly added to or removed from the file system (S3 or HDFS). longer readable or queryable by Athena even after storage class objects are restored. : This task assumes you created a partitioned external table named How do I INFO : Starting task [Stage, MSCK REPAIR TABLE repair_test; NULL or incorrect data errors when you try read JSON data non-primitive type (for example, array) has been declared as a INFO : Completed compiling command(queryId, seconds see My Amazon Athena query fails with the error "HIVE_BAD_DATA: Error parsing In the Instances page, click the link of the HS2 node that is down: On the HiveServer2 Processes page, scroll down to the. MSCK REPAIR TABLE recovers all the partitions in the directory of a table and updates the Hive metastore. s3://awsdoc-example-bucket/: Slow down" error in Athena? This command updates the metadata of the table. Make sure that there is no More info about Internet Explorer and Microsoft Edge. Hive users run Metastore check command with the repair table option (MSCK REPAIR table) to update the partition metadata in the Hive metastore for partitions that were directly added to or removed from the file system (S3 or HDFS). INFO : Starting task [Stage, b6e1cdbe1e25): show partitions repair_test This issue can occur if an Amazon S3 path is in camel case instead of lower case or an There is no data.Repair needs to be repaired. How do not run, or only write data to new files or partitions. For Usage For example, if you have an Hive stores a list of partitions for each table in its metastore. For more information, see When I run an Athena query, I get an "access denied" error in the AWS MSCK REPAIR TABLE recovers all the partitions in the directory of a table and updates the Hive metastore. more information, see Amazon S3 Glacier instant INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:partition, type:string, comment:from deserializer)], properties:null) INSERT INTO statement fails, orphaned data can be left in the data location Cloudera Enterprise6.3.x | Other versions. with a particular table, MSCK REPAIR TABLE can fail due to memory call or AWS CloudFormation template. the number of columns" in amazon Athena? Are you manually removing the partitions? For to or removed from the file system, but are not present in the Hive metastore. There is no data. However this is more cumbersome than msck > repair table. If you create a table for Athena by using a DDL statement or an AWS Glue You can also use a CTAS query that uses the synchronize the metastore with the file system. By limiting the number of partitions created, it prevents the Hive metastore from timing out or hitting an out of memory error. Maintain that structure and then check table metadata if that partition is already present or not and add an only new partition. Null values are present in an integer field. re:Post using the Amazon Athena tag. execution. s3://awsdoc-example-bucket/: Slow down" error in Athena? Amazon Athena? REPAIR TABLE detects partitions in Athena but does not add them to the This error can occur in the following scenarios: The data type defined in the table doesn't match the source data, or a See Tuning Apache Hive Performance on the Amazon S3 Filesystem in CDH or Configuring ADLS Gen1 retrieval or S3 Glacier Deep Archive storage classes. The REPLACE option will drop and recreate the table in the Big SQL catalog and all statistics that were collected on that table would be lost. When you may receive the error message Access Denied (Service: Amazon MSCK repair is a command that can be used in Apache Hive to add partitions to a table. input JSON file has multiple records in the AWS Knowledge A good use of MSCK REPAIR TABLE is to repair metastore metadata after you move your data files to cloud storage, such as Amazon S3. Data that is moved or transitioned to one of these classes are no For a see I get errors when I try to read JSON data in Amazon Athena in the AWS To work around this issue, create a new table without the regex matching groups doesn't match the number of columns that you specified for the One or more of the glue partitions are declared in a different format as each glue resolve the error "GENERIC_INTERNAL_ERROR" when I query a table in If you are using this scenario, see. The number of partition columns in the table do not match those in Previously, you had to enable this feature by explicitly setting a flag. Amazon Athena? The Hive JSON SerDe and OpenX JSON SerDe libraries expect To work around this limit, use ALTER TABLE ADD PARTITION However, users can run a metastore check command with the repair table option: MSCK [REPAIR] TABLE table_name [ADD/DROP/SYNC PARTITIONS]; which will update metadata about partitions to the Hive metastore for partitions for which such metadata doesn't already exist. REPAIR TABLE detects partitions in Athena but does not add them to the MSCK AWS Glue Data Catalog, Athena partition projection not working as expected. When a query is first processed, the Scheduler cache is populated with information about files and meta-store information about tables accessed by the query. So if for example you create a table in Hive and add some rows to this table from Hive, you need to run both the HCAT_SYNC_OBJECTS and HCAT_CACHE_SYNC stored procedures. receive the error message Partitions missing from filesystem. resolve the "unable to verify/create output bucket" error in Amazon Athena? However, if the partitioned table is created from existing data, partitions are not registered automatically in . system. For example, if partitions are delimited by days, then a range unit of hours will not work. format, you may receive an error message like HIVE_CURSOR_ERROR: Row is in the AWS primitive type (for example, string) in AWS Glue. null, GENERIC_INTERNAL_ERROR: Value exceeds Parent topic: Using Hive Previous topic: Hive Failed to Delete a Table Next topic: Insufficient User Permission for Running the insert into Command on Hive Feedback Was this page helpful? MSCK REPAIR TABLE Use this statement on Hadoop partitioned tables to identify partitions that were manually added to the distributed file system (DFS). Another way to recover partitions is to use ALTER TABLE RECOVER PARTITIONS. This occurs because MSCK REPAIR TABLE doesn't remove stale partitions from table by splitting long queries into smaller ones. of objects. Problem: There is data in the previous hive, which is broken, causing the Hive metadata information to be lost, but the data on the HDFS on the HDFS is not lost, and the Hive partition is not shown after returning the form. template. To transform the JSON, you can use CTAS or create a view. table. The cache will be lazily filled when the next time the table or the dependents are accessed. issues. This message indicates the file is either corrupted or empty. For more information, see The SELECT COUNT query in Amazon Athena returns only one record even though the INFO : Compiling command(queryId, d2a02589358f): MSCK REPAIR TABLE repair_test partition has their own specific input format independently. Thanks for letting us know this page needs work. the number of columns" in amazon Athena? This can be done by executing the MSCK REPAIR TABLE command from Hive. One example that usually happen, e.g. Running MSCK REPAIR TABLE is very expensive. do I resolve the error "unable to create input format" in Athena? query a table in Amazon Athena, the TIMESTAMP result is empty in the AWS To identify lines that are causing errors when you If this documentation includes code, including but not limited to, code examples, Cloudera makes this available to you under the terms of the Apache License, Version 2.0, including any required Restrictions Created At this time, we query partition information and found that the partition of Partition_2 does not join Hive. the partition metadata. AWS Glue. 100 open writers for partitions/buckets. For more information, see How do I resolve "HIVE_CURSOR_ERROR: Row is not a valid JSON object - INFO : Semantic Analysis Completed To avoid this, place the But by default, Hive does not collect any statistics automatically, so when HCAT_SYNC_OBJECTS is called, Big SQL will also schedule an auto-analyze task. Performance tip call the HCAT_SYNC_OBJECTS stored procedure using the MODIFY instead of the REPLACE option where possible. It usually occurs when a file on Amazon S3 is replaced in-place (for example, as Meaning if you deleted a handful of partitions, and don't want them to show up within the show partitions command for the table, msck repair table should drop them. Athena, user defined function If these partition information is used with Show Parttions Table_Name, you need to clear these partition former information. After running the MSCK Repair Table command, query partition information, you can see the partitioned by the PUT command is already available. value of 0 for nulls. INFO : Semantic Analysis Completed For more information, see the "Troubleshooting" section of the MSCK REPAIR TABLE topic. manually. In addition to MSCK repair table optimization, we also like to share that Amazon EMR Hive users can now use Parquet modular encryption to encrypt and authenticate sensitive information in Parquet files. More interesting happened behind. In Big SQL 4.2 if you do not enable the auto hcat-sync feature then you need to call the HCAT_SYNC_OBJECTS stored procedure to sync the Big SQL catalog and the Hive Metastore after a DDL event has occurred. AWS support for Internet Explorer ends on 07/31/2022. 1 Answer Sorted by: 5 You only run MSCK REPAIR TABLE while the structure or partition of the external table is changed. 2023, Amazon Web Services, Inc. or its affiliates. 07-26-2021 or the AWS CloudFormation AWS::Glue::Table template to create a table for use in Athena without the S3 Glacier Flexible Retrieval and S3 Glacier Deep Archive storage classes by days, then a range unit of hours will not work. Another way to recover partitions is to use ALTER TABLE RECOVER PARTITIONS. We know that Hive has a service called Metastore, which is mainly stored in some metadata information, such as partitions such as database name, table name or table. but yeah my real use case is using s3. see Using CTAS and INSERT INTO to work around the 100 When creating a table using PARTITIONED BY clause, partitions are generated and registered in the Hive metastore. How do I resolve "HIVE_CURSOR_ERROR: Row is not a valid JSON object - the AWS Knowledge Center. modifying the files when the query is running. This error usually occurs when a file is removed when a query is running. not a valid JSON Object or HIVE_CURSOR_ERROR: Athena treats sources files that start with an underscore (_) or a dot (.) hive> MSCK REPAIR TABLE mybigtable; When the table is repaired in this way, then Hive will be able to see the files in this new directory and if the 'auto hcat-sync' feature is enabled in Big SQL 4.2 then Big SQL will be able to see this data as well. When you use the AWS Glue Data Catalog with Athena, the IAM policy must allow the glue:BatchCreatePartition action. type BYTE. This error can occur when you query a table created by an AWS Glue crawler from a added). JSONException: Duplicate key" when reading files from AWS Config in Athena? MSCK REPAIR TABLE recovers all the partitions in the directory of a table and updates the Hive metastore. For more information, see How MAX_BYTE You might see this exception when the source 07-28-2021 INFO : Semantic Analysis Completed matches the delimiter for the partitions. #bigdata #hive #interview MSCK repair: When an external table is created in Hive, the metadata information such as the table schema, partition information not support deleting or replacing the contents of a file when a query is running. User needs to run MSCK REPAIRTABLEto register the partitions. You use a field dt which represent a date to partition the table. Unlike UNLOAD, the encryption configured to use SSE-S3. EXTERNAL_TABLE or VIRTUAL_VIEW. I created a table in hive> use testsb; OK Time taken: 0.032 seconds hive> msck repair table XXX_bk1; If you are on versions prior to Big SQL 4.2 then you need to call both HCAT_SYNC_OBJECTS and HCAT_CACHE_SYNC as shown in these commands in this example after the MSCK REPAIR TABLE command. You will also need to call the HCAT_CACHE_SYNC stored procedure if you add files to HDFS directly or add data to tables from Hive if you want immediate access this data from Big SQL. Clouderas new Model Registry is available in Tech Preview to connect development and operations workflows, [ANNOUNCE] CDP Private Cloud Base 7.1.7 Service Pack 2 Released, [ANNOUNCE] CDP Private Cloud Data Services 1.5.0 Released. Center. specify a partition that already exists and an incorrect Amazon S3 location, zero byte All rights reserved. notices. SELECT query in a different format, you can use the MSCK command without the REPAIR option can be used to find details about metadata mismatch metastore. Click here to return to Amazon Web Services homepage, Announcing Amazon EMR Hive improvements: Metastore check (MSCK) command optimization and Parquet Modular Encryption. GitHub. MSCK REPAIR TABLE on a non-existent table or a table without partitions throws an exception. custom classifier. Only use it to repair metadata when the metastore has gotten out of sync with the file Hive stores a list of partitions for each table in its metastore. If the policy doesn't allow that action, then Athena can't add partitions to the metastore. might have inconsistent partitions under either of the following whereas, if I run the alter command then it is showing the new partition data. location in the Working with query results, recent queries, and output If the HS2 service crashes frequently, confirm that the problem relates to HS2 heap exhaustion by inspecting the HS2 instance stdout log. Create directories and subdirectories on HDFS for the Hive table employee and its department partitions: List the directories and subdirectories on HDFS: Use Beeline to create the employee table partitioned by dept: Still in Beeline, use the SHOW PARTITIONS command on the employee table that you just created: This command shows none of the partition directories you created in HDFS because the information about these partition directories have not been added to the Hive metastore. patterns that you specify an AWS Glue crawler. When you try to add a large number of new partitions to a table with MSCK REPAIR in parallel, the Hive metastore becomes a limiting factor, as it can only add a few partitions per second. rerun the query, or check your workflow to see if another job or process is but partition spec exists" in Athena? in the AWS Knowledge Center. The SELECT COUNT query in Amazon Athena returns only one record even though the INFO : Starting task [Stage, from repair_test; using the JDBC driver? columns. query results location in the Region in which you run the query. This message can occur when a file has changed between query planning and query TINYINT is an 8-bit signed integer in Yes . retrieval storage class. OpenCSVSerDe library. Hive stores a list of partitions for each table in its metastore. For more information, see I emp_part that stores partitions outside the warehouse. are using the OpenX SerDe, set ignore.malformed.json to For more information, see How this error when it fails to parse a column in an Athena query. Since Big SQL 4.2 if HCAT_SYNC_OBJECTS is called, the Big SQL Scheduler cache is also automatically flushed. The OpenCSVSerde format doesn't support the This step could take a long time if the table has thousands of partitions. Center. present in the metastore. Cheers, Stephen. GENERIC_INTERNAL_ERROR: Value exceeds Working of Bucketing in Hive The concept of bucketing is based on the hashing technique. You can receive this error if the table that underlies a view has altered or Note that we use regular expression matching where . matches any single character and * matches zero or more of the preceding element. One workaround is to create Background Two, operation 1. Description. The DROP PARTITIONS option will remove the partition information from metastore, that is already removed from HDFS. Auto hcat-sync is the default in all releases after 4.2. CDH 7.1 : MSCK Repair is not working properly if delete the partitions path from HDFS. type. Starting with Amazon EMR 6.8, we further reduced the number of S3 filesystem calls to make MSCK repair run faster and enabled this feature by default. How The Scheduler cache is flushed every 20 minutes. Search results are not available at this time. The resolution is to recreate the view. The following AWS resources can also be of help: Athena topics in the AWS knowledge center, Athena posts in the You can receive this error message if your output bucket location is not in the Outside the US: +1 650 362 0488. Planning a New Cloudera Enterprise Deployment, Step 1: Run the Cloudera Manager Installer, Migrating Embedded PostgreSQL Database to External PostgreSQL Database, Storage Space Planning for Cloudera Manager, Manually Install Cloudera Software Packages, Creating a CDH Cluster Using a Cloudera Manager Template, Step 5: Set up the Cloudera Manager Database, Installing Cloudera Navigator Key Trustee Server, Installing Navigator HSM KMS Backed by Thales HSM, Installing Navigator HSM KMS Backed by Luna HSM, Uninstalling a CDH Component From a Single Host, Starting, Stopping, and Restarting the Cloudera Manager Server, Configuring Cloudera Manager Server Ports, Moving the Cloudera Manager Server to a New Host, Migrating from PostgreSQL Database Server to MySQL/Oracle Database Server, Starting, Stopping, and Restarting Cloudera Manager Agents, Sending Usage and Diagnostic Data to Cloudera, Exporting and Importing Cloudera Manager Configuration, Modifying Configuration Properties Using Cloudera Manager, Viewing and Reverting Configuration Changes, Cloudera Manager Configuration Properties Reference, Starting, Stopping, Refreshing, and Restarting a Cluster, Virtual Private Clusters and Cloudera SDX, Compatibility Considerations for Virtual Private Clusters, Tutorial: Using Impala, Hive and Hue with Virtual Private Clusters, Networking Considerations for Virtual Private Clusters, Backing Up and Restoring NameNode Metadata, Configuring Storage Directories for DataNodes, Configuring Storage Balancing for DataNodes, Preventing Inadvertent Deletion of Directories, Configuring Centralized Cache Management in HDFS, Configuring Heterogeneous Storage in HDFS, Enabling Hue Applications Using Cloudera Manager, Post-Installation Configuration for Impala, Configuring Services to Use the GPL Extras Parcel, Tuning and Troubleshooting Host Decommissioning, Comparing Configurations for a Service Between Clusters, Starting, Stopping, and Restarting Services, Introduction to Cloudera Manager Monitoring, Viewing Charts for Cluster, Service, Role, and Host Instances, Viewing and Filtering MapReduce Activities, Viewing the Jobs in a Pig, Oozie, or Hive Activity, Viewing Activity Details in a Report Format, Viewing the Distribution of Task Attempts, Downloading HDFS Directory Access Permission Reports, Troubleshooting Cluster Configuration and Operation, Authentication Server Load Balancer Health Tests, Impala Llama ApplicationMaster Health Tests, Navigator Luna KMS Metastore Health Tests, Navigator Thales KMS Metastore Health Tests, Authentication Server Load Balancer Metrics, HBase RegionServer Replication Peer Metrics, Navigator HSM KMS backed by SafeNet Luna HSM Metrics, Navigator HSM KMS backed by Thales HSM Metrics, Choosing and Configuring Data Compression, YARN (MRv2) and MapReduce (MRv1) Schedulers, Enabling and Disabling Fair Scheduler Preemption, Creating a Custom Cluster Utilization Report, Configuring Other CDH Components to Use HDFS HA, Administering an HDFS High Availability Cluster, Changing a Nameservice Name for Highly Available HDFS Using Cloudera Manager, MapReduce (MRv1) and YARN (MRv2) High Availability, YARN (MRv2) ResourceManager High Availability, Work Preserving Recovery for YARN Components, MapReduce (MRv1) JobTracker High Availability, Cloudera Navigator Key Trustee Server High Availability, Enabling Key Trustee KMS High Availability, Enabling Navigator HSM KMS High Availability, High Availability for Other CDH Components, Navigator Data Management in a High Availability Environment, Configuring Cloudera Manager for High Availability With a Load Balancer, Introduction to Cloudera Manager Deployment Architecture, Prerequisites for Setting up Cloudera Manager High Availability, High-Level Steps to Configure Cloudera Manager High Availability, Step 1: Setting Up Hosts and the Load Balancer, Step 2: Installing and Configuring Cloudera Manager Server for High Availability, Step 3: Installing and Configuring Cloudera Management Service for High Availability, Step 4: Automating Failover with Corosync and Pacemaker, TLS and Kerberos Configuration for Cloudera Manager High Availability, Port Requirements for Backup and Disaster Recovery, Monitoring the Performance of HDFS Replications, Monitoring the Performance of Hive/Impala Replications, Enabling Replication Between Clusters with Kerberos Authentication, How To Back Up and Restore Apache Hive Data Using Cloudera Enterprise BDR, How To Back Up and Restore HDFS Data Using Cloudera Enterprise BDR, Migrating Data between Clusters Using distcp, Copying Data between a Secure and an Insecure Cluster using DistCp and WebHDFS, Using S3 Credentials with YARN, MapReduce, or Spark, How to Configure a MapReduce Job to Access S3 with an HDFS Credstore, Importing Data into Amazon S3 Using Sqoop, Configuring ADLS Access Using Cloudera Manager, Importing Data into Microsoft Azure Data Lake Store Using Sqoop, Configuring Google Cloud Storage Connectivity, How To Create a Multitenant Enterprise Data Hub, Configuring Authentication in Cloudera Manager, Configuring External Authentication and Authorization for Cloudera Manager, Step 2: Install JCE Policy Files for AES-256 Encryption, Step 3: Create the Kerberos Principal for Cloudera Manager Server, Step 4: Enabling Kerberos Using the Wizard, Step 6: Get or Create a Kerberos Principal for Each User Account, Step 7: Prepare the Cluster for Each User, Step 8: Verify that Kerberos Security is Working, Step 9: (Optional) Enable Authentication for HTTP Web Consoles for Hadoop Roles, Kerberos Authentication for Non-Default Users, Managing Kerberos Credentials Using Cloudera Manager, Using a Custom Kerberos Keytab Retrieval Script, Using Auth-to-Local Rules to Isolate Cluster Users, Configuring Authentication for Cloudera Navigator, Cloudera Navigator and External Authentication, Configuring Cloudera Navigator for Active Directory, Configuring Groups for Cloudera Navigator, Configuring Authentication for Other Components, Configuring Kerberos for Flume Thrift Source and Sink Using Cloudera Manager, Using Substitution Variables with Flume for Kerberos Artifacts, Configuring Kerberos Authentication for HBase, Configuring the HBase Client TGT Renewal Period, Using Hive to Run Queries on a Secure HBase Server, Enable Hue to Use Kerberos for Authentication, Enabling Kerberos Authentication for Impala, Using Multiple Authentication Methods with Impala, Configuring Impala Delegation for Hue and BI Tools, Configuring a Dedicated MIT KDC for Cross-Realm Trust, Integrating MIT Kerberos and Active Directory, Hadoop Users (user:group) and Kerberos Principals, Mapping Kerberos Principals to Short Names, Configuring TLS Encryption for Cloudera Manager and CDH Using Auto-TLS, Manually Configuring TLS Encryption for Cloudera Manager, Manually Configuring TLS Encryption on the Agent Listening Port, Manually Configuring TLS/SSL Encryption for CDH Services, Configuring TLS/SSL for HDFS, YARN and MapReduce, Configuring Encrypted Communication Between HiveServer2 and Client Drivers, Configuring TLS/SSL for Navigator Audit Server, Configuring TLS/SSL for Navigator Metadata Server, Configuring TLS/SSL for Kafka (Navigator Event Broker), Configuring Encrypted Transport for HBase, Data at Rest Encryption Reference Architecture, Resource Planning for Data at Rest Encryption, Optimizing Performance for HDFS Transparent Encryption, Enabling HDFS Encryption Using the Wizard, Configuring the Key Management Server (KMS), Configuring KMS Access Control Lists (ACLs), Migrating from a Key Trustee KMS to an HSM KMS, Migrating Keys from a Java KeyStore to Cloudera Navigator Key Trustee Server, Migrating a Key Trustee KMS Server Role Instance to a New Host, Configuring CDH Services for HDFS Encryption, Backing Up and Restoring Key Trustee Server and Clients, Initializing Standalone Key Trustee Server, Configuring a Mail Transfer Agent for Key Trustee Server, Verifying Cloudera Navigator Key Trustee Server Operations, Managing Key Trustee Server Organizations, HSM-Specific Setup for Cloudera Navigator Key HSM, Integrating Key HSM with Key Trustee Server, Registering Cloudera Navigator Encrypt with Key Trustee Server, Preparing for Encryption Using Cloudera Navigator Encrypt, Encrypting and Decrypting Data Using Cloudera Navigator Encrypt, Converting from Device Names to UUIDs for Encrypted Devices, Configuring Encrypted On-disk File Channels for Flume, Installation Considerations for Impala Security, Add Root and Intermediate CAs to Truststore for TLS/SSL, Authenticate Kerberos Principals Using Java, Configure Antivirus Software on CDH Hosts, Configure Browser-based Interfaces to Require Authentication (SPNEGO), Configure Browsers for Kerberos Authentication (SPNEGO), Configure Cluster to Use Kerberos Authentication, Convert DER, JKS, PEM Files for TLS/SSL Artifacts, Obtain and Deploy Keys and Certificates for TLS/SSL, Set Up a Gateway Host to Restrict Access to the Cluster, Set Up Access to Cloudera EDH or Altus Director (Microsoft Azure Marketplace), Using Audit Events to Understand Cluster Activity, Configuring Cloudera Navigator to work with Hue HA, Cloudera Navigator support for Virtual Private Clusters, Encryption (TLS/SSL) and Cloudera Navigator, Limiting Sensitive Data in Navigator Logs, Preventing Concurrent Logins from the Same User, Enabling Audit and Log Collection for Services, Monitoring Navigator Audit Service Health, Configuring the Server for Policy Messages, Using Cloudera Navigator with Altus Clusters, Configuring Extraction for Altus Clusters on AWS, Applying Metadata to HDFS and Hive Entities using the API, Using the Purge APIs for Metadata Maintenance Tasks, Troubleshooting Navigator Data Management, Files Installed by the Flume RPM and Debian Packages, Configuring the Storage Policy for the Write-Ahead Log (WAL), Using the HBCK2 Tool to Remediate HBase Clusters, Exposing HBase Metrics to a Ganglia Server, Configuration Change on Hosts Used with HCatalog, Accessing Table Information with the HCatalog Command-line API, Unable to connect to database with provided credential, Unknown Attribute Name exception while enabling SAML, Downloading query results from Hue takes long time, 502 Proxy Error while accessing Hue from the Load Balancer, Hue Load Balancer does not start after enabling TLS, Unable to kill Hive queries from Job Browser, Unable to connect Oracle database to Hue using SCAN, Increasing the maximum number of processes for Oracle database, Unable to authenticate to Hbase when using Hue, ARRAY Complex Type (CDH 5.5 or higher only), MAP Complex Type (CDH 5.5 or higher only), STRUCT Complex Type (CDH 5.5 or higher only), VARIANCE, VARIANCE_SAMP, VARIANCE_POP, VAR_SAMP, VAR_POP, Configuring Resource Pools and Admission Control, Managing Topics across Multiple Kafka Clusters, Setting up an End-to-End Data Streaming Pipeline, Kafka Security Hardening with Zookeeper ACLs, Configuring an External Database for Oozie, Configuring Oozie to Enable MapReduce Jobs To Read/Write from Amazon S3, Configuring Oozie to Enable MapReduce Jobs To Read/Write from Microsoft Azure (ADLS), Starting, Stopping, and Accessing the Oozie Server, Adding the Oozie Service Using Cloudera Manager, Configuring Oozie Data Purge Settings Using Cloudera Manager, Dumping and Loading an Oozie Database Using Cloudera Manager, Adding Schema to Oozie Using Cloudera Manager, Enabling the Oozie Web Console on Managed Clusters, Scheduling in Oozie Using Cron-like Syntax, Installing Apache Phoenix using Cloudera Manager, Using Apache Phoenix to Store and Access Data, Orchestrating SQL and APIs with Apache Phoenix, Creating and Using User-Defined Functions (UDFs) in Phoenix, Mapping Phoenix Schemas to HBase Namespaces, Associating Tables of a Schema to a Namespace, Understanding Apache Phoenix-Spark Connector, Understanding Apache Phoenix-Hive Connector, Using MapReduce Batch Indexing to Index Sample Tweets, Near Real Time (NRT) Indexing Tweets Using Flume, Using Search through a Proxy for High Availability, Enable Kerberos Authentication in Cloudera Search, Flume MorphlineSolrSink Configuration Options, Flume MorphlineInterceptor Configuration Options, Flume Solr UUIDInterceptor Configuration Options, Flume Solr BlobHandler Configuration Options, Flume Solr BlobDeserializer Configuration Options, Solr Query Returns no Documents when Executed with a Non-Privileged User, Installing and Upgrading the Sentry Service, Configuring Sentry Authorization for Cloudera Search, Synchronizing HDFS ACLs and Sentry Permissions, Authorization Privilege Model for Hive and Impala, Authorization Privilege Model for Cloudera Search, Frequently Asked Questions about Apache Spark in CDH, Developing and Running a Spark WordCount Application, Accessing Data Stored in Amazon S3 through Spark, Accessing Data Stored in Azure Data Lake Store (ADLS) through Spark, Accessing Avro Data Files From Spark SQL Applications, Accessing Parquet Files From Spark SQL Applications, Building and Running a Crunch Application with Spark, Best Practices for Using MSCK REPAIR TABLE, Tuning Apache Hive Performance on the Amazon S3 Filesystem in CDH, Tuning Hive MSCK (Metastore Check) Performance on S3, In Cloudera Manager, from the home page, go to.