Data Partitioning and Sharding Questions and Answers for Viva

Answer-1: Using replication, failover mechanisms, and data redundancy across nodes.

Answer-2: It may need to be split, rebalanced, or moved to another node for load distribution.

Answer-3: Consistent hashing distributes data across nodes such that minimal reorganization is needed when nodes are added or removed.

Answer-4: They are complex and slow because data must be retrieved and combined from multiple shards.

Answer-5: It highlights trade-offs between Consistency, Availability, and Partition tolerance in distributed systems.

Answer-6: A model where updates to data propagate to all nodes over time, rather than immediately.

Answer-7: Through constraints, application-level logic, and database mechanisms like transactions and replication.

Answer-8: It refers to adding or removing shards dynamically to adapt to changing workloads.

Answer-9: Each shard may serve a different customer (tenant), isolating their data for scalability and security.

Answer-10: Tools like MongoDB, Vitess (for MySQL), Citus (for PostgreSQL), and Cosmos DB support native or plugin-based sharding.

Answer-11: Data partitioning is the process of dividing a database into distinct, independent parts to improve performance, manageability, and scalability.

Answer-12: Sharding is a type of partitioning that distributes data across multiple machines or database instances.

Answer-13: The main goal is to scale out a database system to handle large amounts of data and high throughput by distributing the load.

Answer-14: Horizontal partitioning splits rows into different tables, while vertical partitioning splits columns into different tables.

Answer-15: A shard key is the field used to determine how data is distributed across shards.

Answer-16: Range-based sharding divides data into shards based on continuous ranges of the shard key.

Answer-17: Hash-based sharding uses a hash function on the shard key to distribute data evenly across shards.

Answer-18: Directory-based sharding uses a lookup table to map each key to its corresponding shard.

Answer-19: Sharding improves performance, enables horizontal scaling, and enhances fault isolation.

Answer-20: Challenges include increased complexity, rebalancing data, maintaining consistency, and cross-shard queries.

Answer-21: Re-sharding is the process of redistributing data among shards, typically needed when a shard becomes too large or hot.

Answer-22: A hot shard is one that receives disproportionately high traffic, causing performance bottlenecks.

Answer-23: Auto-sharding automatically distributes data and manages shards behind the scenes without user intervention.

Answer-24: A shard map keeps track of which data resides on which shard.

Answer-25: Vertical sharding splits a database schema into different logical sections and distributes them across shards.

Answer-26: A good shard key ensures even distribution, minimizes cross-shard operations, and aligns with query patterns.

Answer-27: A cross-shard query is one that requires data from multiple shards, which can degrade performance.

Answer-28: It can be handled using distributed transactions, eventual consistency, or replication mechanisms.

Answer-29: Yes, SQL databases like MySQL and PostgreSQL can be sharded, though it may require manual configuration or third-party tools.

Answer-30: Yes, most NoSQL databases like MongoDB, Cassandra, and HBase support sharding natively.

Answer-31: A partition key is the field used to determine how data is grouped within a partition.

Answer-32: Partitioning allows queries to scan only relevant partitions, reducing I/O and improving speed.

Answer-33: Partitioning can occur within a single database instance, while sharding distributes data across multiple instances or nodes.

Answer-34: Composite shard keys consist of multiple fields to provide better distribution and query support.

Answer-35: Cloud databases often provide built-in auto-sharding and transparent partitioning to scale with demand.

Answer-36: It refers to a system where multiple databases appear as a single logical database but operate independently.

Answer-37: It allows indexing of fields other than the shard key to support efficient queries across shards.

Answer-38: It divides data based on continuous ranges of values, such as date ranges or ID intervals.

Answer-39: List partitioning uses predefined lists of values to distribute rows among partitions.

Answer-40: Hash partitioning uses a hash function on one or more columns to determine the target partition.

Answer-41: Composite partitioning combines two partitioning methods, such as range-hash or range-list.

Answer-42: Sub-partitioning is applying a second level of partitioning within each primary partition.

Answer-43: A partitioned table is a table whose data is physically separated into multiple storage segments or partitions.

Answer-44: Metadata stores the mapping of shard keys to shard locations and helps route queries appropriately.

Answer-45: Logical partitioning is how data is organized conceptually, while physical partitioning refers to actual storage layout.

Answer-46: Monitoring involves tracking shard health, load balancing, replication lag, and query performance.

Answer-47: Key-based routing uses the shard key to route queries directly to the correct shard.

Answer-48: Drawbacks include operational complexity, difficult joins, and challenges in global transactions.

Answer-49: Yes, local and global indexes can be used depending on the database system and use case.

Answer-50: Partition pruning eliminates irrelevant partitions during query execution to improve performance.