Posts in Postgres

postgres_fdw estimated overhead

Concept In the current concept, we are going to combine Foreign tables inheritance with the postgres_fdw extension, both being already available features since 9.5 version. Cross-node partitioning allows a better data locality and a more scalable model than keeping local partitions. Being said, the data will be split into several nodes and organized using a particular key, which will determine in which shard data will be allocated. For the current POC, we are going to specify the shardKey , which is a simple char(2) type.

Simple and manual sharding on PostgreSQL.

Before starting the container This article is not an introductory explanation of docker,however it’s scope if for docker’s beginners. You can consider it as an extension of the well documented Percona docker hub doc. For the source code of the image, the repository is at github. Here is the all what you need to do for start: 1 docker run --name percona57 -e MYSQL_ROOT_PASSWORD=<a_password> -d percona:5.7 For checking the container status log, you can execute docker logs percona57.

Connecting Postgres and Kafka rawly

Apache Kafka and Postgres: Transaction and reporting capabilities Apache Kafka is a well known distributed streaming platform for data processing and consistent messaging. It allows you to consistently centralize data streams for several purposes by consuming and producing them. One of the examples of a nice implementation, is the Mozilla’s Data pipeline implementation, particularly as it shows Kafka as an entry point of the data flow. This allows you to plug new data stores bellow its stream, making it easy to use different data store formats ( such as DRBMS or Document, etc.

Highlighting Postgres 10 new features: Logical Replication and Partitioning.

Heya! I this article we are going to explore two of the major features commited in the upcoming PostgreSQL release: Logical Replication and Partitioning. Needeless to say that these features aren’t yet available in the stable release, so they are prune to change or extended. Advertising warning! The current article is just a sneak peak of the upcoming talk Demystifying Logical Replication on PostgreSQL at Percona Live Santa Clara 2017. Get your tickets!

PostgreSQL RDS pg-stat-ramdisk-size new feature and its calculations

IMPORTANT NOTE This has been already addressed in PostgreSQL core, but this option is still available in RDS. What does it change and why is so important? Tracking databases and not just tables counters in Postgres isn’t cheap, but since some time ago there were workarounds involving the setup of a ramdisk to place the directory pointed by stat_temp_directory GUC variable. That directory places a global.stat and a per-database stat files called like db_<oidOfDB>.

Random data for PostgreSQL

Usually, you want to test code, try new features or just need decent content when generating random data. Random databases for Postgres addresses this need by simple SQL scripts that can be entirely generated inside your database. It isn’t a sophisticated repository, although it is practical for quick usage. What has inside? Pure SQL If you want to indicate issues, go for it at report issues.

Multi source data injection to Postgres RDS with encryption and FTS support

Sponsored: Pythian Inc. Note 1: All the of this presentation is published in this repository. You will find a lot of folders and information, probably part of a blog series. Note 2: All the work on this article is a POC (Proof of concept). Note 3: This is something that is related for HIPAA compliant. KMS/RDS The POC on this article was developed before the releasing of the Key Management service for RDS.