Организация физического уровня колоночных СУБД

Георгий Алексеевич Чернышев

doi:10.15622/sp.30.13

Георгий Алексеевич Чернышев ассистент кафедры информатики математико-механического факультета Санкт-Петербургский государственный университет (СПбГУ)

DOI:

https://doi.org/10.15622/sp.30.13

Ключевые слова:

базы данных, колоночные СУБД, физический уровень СУБД, настройка СУБД, вертикальное фрагментирование, горизонтальное фрагментирование, размещение данных

Аннотация

данной работе мы рассмотрели колоночные СУБД и их устройство. Было дано определение колоночной СУБД, представлены отличия от классических СУБД с построчным хранением. Были представлены сильные и слабые стороны колоночного подхода, причины его появления и успеха. Кроме того, в работе описана ниша, занимаемая современными коммерческими колоночными СУБД. Затем нами был рассмотрен набор технологических решений, используемых в данных СУБД. Наконец, в данной работе были рассмотрены вопросы выбора структур физического уровня для колоночных СУБД. В начале представлено краткое введение в проблему выбора данных структур. Затем, на основе произведенного обзора анализируется применимость подходов к устройству физического уровня классических СУБД (фрагментирование, размещение и пр.) к колоночным СУБД. Кроме того, рассматриваются способы, характерные именно для колоночных систем (различные порядки колонок, индексы соединения).

Литература

Кузнецов С.Д. Год эпохи перемен в технологии баз данных // Труды Института системного программирования РАН. 2010. №19. С. 9-34

Кузнецов С.Д. MapReduce: внутри, снаружи или сбоку от параллельных СУБД? // Труды Института системного программирования РАН. 2010. № 19. С. 35-70

Чернышев Г.А. Обзор подходов к организации физического уровня в СУБД // Тр. СПИИРАН. 2013. Вып. 1(24). С. 222-277

Abadi D., Madden S., Ferreira M. Integrating compression and execution in column-oriented database systems // Proceedings of the 2006 ACM SIGMOD international conference on Management of data, SIGMOD '06 (Chicago, IL, USA). New York, NY, USA: ACM, 2006. P. 671-682

Abadi D.J., Boncz P.A., Harizopoulos S. Column-oriented database systems // Proc. VLDB Endow., VLDB'09. VLDB Endowment, 2009. P. 1664-1665

Abadi D.J., Madden S.R., Hachem N. Column-stores vs. row-stores: how different are they really? // Proceedings of the 2008 ACM SIGMOD international conference on Management of data, SIGMOD '08 (Vancouver, Canada). New York, NY, USA: ACM, 2008. P. 967-980. ISBN 978-1-60558-102-6

Dageville B., Das D., Dias K., Yagoub K., Zait M., Ziauddin M. Automatic SQL tuning in oracle 10g // Proceedings of the Thirtieth international conference on Very large data bases - Volume 30, VLDB '04 (Toronto, Canada). VLDB Endowment, 2004. P. 1098-1109. ISBN 0-12-088469-0

Bellatreche L., Benkrid S. A Joint Design Approach of Partitioning and Allocation in Parallel Data Warehouses // Data Warehousing and Knowledge Discovery. Lecture Notes in Computer Science. 2009. Vol. 5691. P. 99-110. ISBN 978-3-642-03729-0

Boncz P.A., Kersten M.L., Manegold S. Breaking the memory wall in MonetDB // Commun. ACM. 2008. Vol. 51. No. 12. P. 77-85

Stonebraker M., Abadi D.J., Batkin A., Chen X., Cherniack M., Ferreira M., Lau E., Lin A., Madden S., O'Neil E., O'Neil P., Rasin A., Tran N., Zdonik S. C-store: a column-oriented DBMS // Proceedings of the 31st international conference on Very large data bases, VLDB '05. (Trondheim, Norway). VLDB Endowment, 2005. P. 553-564. ISBN 1-59593-154-6

Chu W.W., Ieong I.T. A transaction-based approach to vertical partitioning for relational database systems // IEEE Transactions on Software Engineering. 1993. Vol. 19. No. 8. P. 804-812

Shao M., Schindler J., Schlosser S.W., Ailamaki A., Ganger G.R. Clotho: decoupling memory page layout from storage organization // Proceedings of the 30th international conference on Very large data bases, VLDB '04. (Toronto, Canada). VLDB Endowment, 2004. P. 696-707. ISBN 0-12-088469-0

Zukowski M., Heman S., Nes N., Boncz P. Cooperative scans: dynamic bandwidth sharing in a DBMS // Proceedings of the 33rd international conference on Very large data bases, VLDB '07 (Vienna, Austria). VLDB Endowment, 2007. P. 723-734. ISBN 978-1-59593-649-3

Copeland G.P., Khoshafian S.N. A decomposition storage model // SIGMOD Rec. 1985. Vol. 14. No. 4. P. 268-279

Cornell D.W., Yu P.S. IEEE Transactions on Software Engineering // An effective approach to vertical partitioning for physical design of relational databases. 1990. Vol. 16. No. 2. P. 248-258

Das S., Agrawal D., El Abbadi A. G-Store: a scalable data store for transactional multi key access in the cloud // Proceedings of the 1st ACM symposium on Cloud computing, SoCC '10 (Indianapolis, Indiana, USA). New York, NY, USA: ACM, 2010. P. 163-174. ISBN 978-1-4503-0036-0

Agrawal S., Chaudhuri S., Kollar L., Marathe A., Narasayya V., Syamala M. Database Tuning Advisor for Microsoft SQL Server 2005 // Proceedings of VLDB, VLDB'04. P. 1110-1121

Agrawal S., Chaudhuri S., Kollar L., Marathe A., Narasayya V., Syamala M. Database tuning advisor for Microsoft SQL server 2005: demo // Proceedings of the 2005 ACM SIGMOD international conference on Management of data, SIGMOD '05 (Baltimore, Maryland). New York, NY, USA: ACM, 2005. P. 930-932. ISBN 1-59593-060-4

Valentin G., Zuliani M., Zilio D.C., Lohman G., Skelley A. DB2 advisor: an optimizer smart enough to recommend its own indexes // Proceedings of 16th International Conference on Data Engineering, 2000, ICDE'00. P. 101-110

Zilio D.C., Rao J., Lightstone S., Lohman G., Storm A., Garcia-Arellano C., Fadden S. DB2 design advisor: integrated automatic physical database design // Proceedings of the Thirtieth international conference on Very large data bases - Volume 30, VLDB'04 (Toronto, Canada). VLDB Endowment, 2004. P. 1087-1097. ISBN 0-12-088469-0

French C.D. "One size fits all" database architectures do not work for DSS // SIGMOD Rec. 1995. Vol. 24. No. 2. P. 449-450

Graefe G. Query evaluation techniques for large databases // ACM Comput. Surv. 1993. Vol. 25. No. 22. P. 73-169

Graefe G., Kuno H. Self-selecting, self-tuning, incrementally optimized indexes // Proceedings of the 13th International Conference on Extending Database Technology, EDBT '10 (Lausanne, Switzerland). New York, NY, USA: ACM, 2010. P. 371-381. ISBN 978-1-60558-945-9

Hankins R.A., Patel J.M. Data morphing: an adaptive, cache-conscious storage technique // Proceedings of the 29th international conference on Very large data bases, VLDB'03 (Berlin, Germany). VLDB Endowment, 2003. P. 417-428. ISBN 0-12-722442-4

Hellerstein J.M., Stonebraker M., Hamilton J. Architecture of a Database System // Found. Trends databases. 2007. Vol. 1. No. 2. P. 141-259

Holloway A.L., Raman V., Swart G., DeWitt D.J. How to barter bits for chronons: compression and bandwidth trade offs for database scans // Proceedings of the 2007 ACM SIGMOD international conference on Management of data, SIGMOD '07 (Beijing, China). New York, NY, USA: ACM, 2007. P. 389-400. ISBN 978-1-59593-686-8

Huang Y-F., Chen J-H. Fragment Allocation in Distributed Database Design // Journal of Information Science and Engineering. 2001. Vol. 17. No. 3. P. 491-506

Schaffner J., Bog A., Kruger J., Zeier A. A Hybrid Row-Column OLTP Database Architecture for Operational Reporting // BIRTE (Informal Proceedings), Informal Proceedings of the Second International Workshop on Business Intelligence for the Real-Time Enterprise, BIRTE 2008, in conjunction with VLDB'08 (Auckland, New Zealand). P. 389-400. ISBN 978-1-59593-686-8

Grund M., Kruger J., Plattner H., Zeier A., Cudre-Mauroux . P., Madden S. HYRISE: a main memory hybrid storage engine // Proc. VLDB Endow., VLDB'10. VLDB Endowment, 2010. P. 105-116

Idreos S., Kersten M.L., Manegold S. Database Cracking // Proceedings of Third Biennial Conference on Innovative Data Systems Research, CIDR 2007 (Asilomar, CA, USA)

Ivanova M., Kersten M.L., Nes N. Self-organizing strategies for a column-store database // Proceedings of the 11th international conference on Extending database technology: Advances in database technology, EDBT '08 (Nantes, France). New York, NY, USA: ACM, 2008. P. 157-168. ISBN 978-1-59593-926-5

Jaecksch B., Lehner W., Faerber F. A plan for OLAP // Proceedings of the 13th International Conference on Extending Database Technology, EDBT '10 (Lausanne, Switzerland). New York, NY, USA: ACM, 2010. P. 681-686. ISBN 978-1-60558-945-9

Karasalo I., Svensson P. The design of cantor: a new system for data analysis // Proceedings of the 3rd international workshop on Statistical and scientific database management. Berkeley, CA, US: Lawrence Berkeley Laboratory, 1986. P. 224-244

Abadi D.J., Myers D.S., DeWitt D.J., Madden S. Materialization Strategies in a Column-Oriented DBMS // Proceedings of the ICDE Conference, ICDE '07. P. 466-475

Nehme R., Bruno N. Automated partitioning design in parallel database systems // Proceedings of the 2011 international conference on Management of data, SIGMOD '11 (Athens, Greece). New York, NY, USA: ACM, 2011. P. 1137-1148. ISBN 978-1-4503-0661-4

OLAP // Encyclopedia of Database Systems. 2009. P. 1947-1947. ISBN 978-0-387-35544-3

O'Neil . P.E., O'Neil E.J., Chen X. The Star Schema Benchmark (SSB). URL: http://www.cs.umb.edu/ poneil/StarSchemaB.PDF

Maier C., Dash D., Alagiannis I., Ailamaki A., Heinis T. PARINDA: an interactive physical designer for PostgreSQL // Proceedings of the 13th International Conference on Extending Database Technology, EDBT '10 (Lausanne, Switzerland). New York, NY, USA: ACM, 2010. P. 701-704. ISBN 978-1-60558-945-9

Harizopoulos S., Liang V., Abadi D.J., Madden S. Performance tradeoffs in read-optimized databases // Proceedings of the 32nd international conference on Very large data bases, VLDB '06 (Seoul, Korea). VLDB Endowment, 2006. P. 487-498

Plattner H. A common database approach for OLTP and OLAP using an in-memory column database // Proceedings of the 2009 ACM SIGMOD International Conference on Management of data, SIGMOD '09 (Providence, Rhode Island, USA). New York, NY, USA: ACM, 2009. P. 1-2. ISBN 978-1-60558-551-2

Eadon G., Chong E.I., Shankar S., Raghavan A., Srinivasan J., Das S. Supporting table partitioning by reference in oracle // Proceedings of the 2008 ACM SIGMOD international conference on Management of data, SIGMOD '08 (Vancouver, Canada). New York, NY, USA: ACM, 2008. P. 1111-1122. ISBN 978-1-60558-102-6

Svensson P. The Evolution of Vertical Database Architectures -- A Historical Review (Keynote Talk) // Proceedings of the 20th international conference on Scientific and Statistical Database Management, SSDBM '08 (Hong Kong, China). Berlin, Heidelberg: Springer-Verlag, 2008. P. 3-5. ISBN 978-3-540-69476-2

Tamhankar A.M., Ram S. Database fragmentation and allocation: an integrated methodology and case study // Trans. Sys. Man Cyber. Part A. 1998. Vol. 28. No. 3. P. 288-305

The Vertica Analytic Database Technical Overview White Paper URL:http://www.vertica.com/wp-content/uploads/2011/01/VerticaArchitectureWhitePaper.pdf

Thiem A., Sattler K-U. An Integrated Approach to Performance Monitoring for Autonomous Tuning // Proceedings of the 2009 IEEE International Conference on Data Engineering, ICDE'09. Washington, DC, USA: IEEE Computer Society, 2009. P. 1671-1678. ISBN 978-0-7695-3545-6

TPC Benchmark H. Decision Support. URL: http://www.tpc.org/tpch

Ailamaki A., DeWitt D.J., Hill M.D., Skounakis M. Weaving Relations for Cache Performance // Proceedings of the 27th International Conference on Very Large Data Bases, VLDB '01. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 2001. P. 169-180. ISBN 1-55860-804-4

Physical design approaches for column-stores

George Chernishev assistant professor of Computer Science Department St.Petersburg State University

Abstract

In this paper we survey column-store database technology. We describe column-store database management system and point out the distinctions from the classical row-store DBMS. We present strong and weak points of the column-stores, the reasons of their appearance and success. We survey a variety of technical solutions, which provide these systems with the advantage. We also describe workloads which benefit this approach the most and thus constitute the area of application of commercial column-store DBMS systems. Then, we present a brief introduction into the problem of DBMS physical design. We analyze the perspectives of various classical approaches (allocation, partitioning etc), as well as column-store specific ones (column sort orders and join index application).

Просмотры	2472
Скачивания	1420

Статьи

Организация физического уровня колоночных СУБД

DOI:

Ключевые слова:

Аннотация

Литература

Physical design approaches for column-stores

Abstract

Опубликован

Статистика

Как цитировать

Выпуск

Раздел

Импакт-фактор

Разделы

Мы в сети

Обратная связь