PostgreSQL La base de donnees la plus sophistiquee au monde.

La planète francophone de PostgreSQL

lundi 14 mai 2012

Actualités PostgreSQL.fr

PG Day France 2012 : une journée de conférences sur le SGBDR PostgreSQL.

Le 7 juin à Lyon se tiendra le PG Day France 2012, une journée de conférences et d'échanges sur le thème du SGBDR open source PostgreSQL. Cette journée sera également l'occasion de rencontrer les acteurs de la communauté PostgreSQL.

Que vous soyez DBA, architecte, développeur, chef de projet utilisant PostgreSQL, vous découvrirez des retours d'expérience d'autres utilisateurs, ainsi que des présentations techniques de PostgreSQL, de PostGIS (cartouche spatiale) ou d'autres extensions. Cette journée est organisée par la communauté francophone des utilisateurs de PostGreSQL, avec le soutien de plusieurs entreprises partenaires (Oslandia, Dalibo, Auriga, EnterpriseDB, 2ndQuadrant).

Inscrivez-vous dès à présent, et retrouvez les informations complémentaires sur le site : http://www.pgday.fr/

Le programme des conférences comporte les sujets suivants :

 * Benchmark Tsung pour PostgreSQL par Cédric Villemain
 * Disponibilité et Durabilité, Architectures et Réplications par Dimitri Fontaine
 * E-Maj ... par l'image par Philippe Beaudouin
 * PostGIS 2.0, la géo nouvelle génération par Vincent Picavet
 * PostgreSQL-f par Grégory SMITS
 * Postgresql, PostGis en collectivité territoriale par Vincent Kober
 * Utilisation de foreign data wrappers dans différents contextes par Ronan Dunklau

Le nombre de places est limité. Inscrivez-vous vite à cette adresse :

http://www.pgday.fr/inscriptions

Rendez-vous à Lyon le 7 juin !

par daamien le lundi 14 mai 2012 à 19h48

Publication de PostgreSQL 9.2 Beta

Le PostgreSQL Global Development Group publie ce jour la version beta de PostgreSQL 9.2. Celle-ci améliore considérablement les performances et les possibilités d'extension verticale et horizontale. L'aide de tous est requise pour tester cette version.

Les avancées principales en terme de performance et d'échelonnabilité incluent :

  • Les parcours des seuls index, qui permettent aux utilisateurs d'éviter d'inefficaces parcours des tables ;
  • la possibilité d'atteindre une charge de lecture sur 64 cœurs de plus de 300 000 requêtes/seconde ;
  • des améliorations de la vitesse d'écriture des données, grâce notamment à la validation de transactions par groupe ;
  • abaissement de la consommation CPU ;
  • réplication en cascade, permettant une distribution géographiques des esclaves.

PostgreSQL 9.2 apporte aussi de nouvelles fonctionnalités pour les développeurs d'application, dont :

  • le support des données JSON, autorisant les bases hybrides document-relationnelle ;
  • les types échelle, ce qui permet de nouveaux types calendrier, échelles de temps, et d'applications analytiques ;
  • de nombreuses améliorations de la commande ALTER, et de quelques autres instructions, facilitant les modifications de bases en production.

La liste complète des fonctionnalités de cette version est disponible sur la page des notes de version : http://www.postgresql.org/docs/devel/static/release-9-2.html

Le projet repose sur la participation active de la communauté à cette période de tests pour permettre la publication d'une version hautement performante et dénuée de bogues.

N'hésitez pas à télécharger PostgreSQL 9.2 béta, à la tester avec vos applications et votre charge, et à faire un retour aux développeurs de PostgreSQL. Vous trouverez toutes les informations sur les tests et les rapports d'erreur à la page : http://www.postgresql.org/developer/beta

Page de téléchargement, binaires, installeurs Windows, Linux et Mac : http://www.postgresql.org/download

La documentation complète, installée avec PostgreSQL, est également disponible en ligne : http://www.postgresql.org/docs/devel/static.

par SAS le lundi 14 mai 2012 à 06h55

lundi 7 mai 2012

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 6 mai 2012

La conférence PostgreSQL China 2012 aura lieu du 14 au 17 juin à Pékin : http://wiki.postgresql.org/wiki/Pgconfchina2012

PostgreSQL Magazine #01 est dans les bacs : http://pgmag.org/01/

Le PUG turque organise la deuxième PgConf turque à Istanbul le 12 mai 2012. Présentation d'ouverture par Magnus Hagander. Inscription gratuite : http://pgday.PostgreSQL.org.tr/2012/

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en mai

PostgreSQL Local

  • La PGCon 2012 sera tenue à l'Université d'Ottawa, les 17 et 18 mai 2012. Elle sera précédée par deux jours de tutoriels les 15 & 16 mai 2012 : http://www.pgcon.org/2012/
  • Le PGDay France aura lieu à Lyon, le 7 juin 2012 : http://www.pgday.fr
  • La PostgreSQL Conference Europe 2012 aura lieu à Prague, République Tchèque, du 23 au 26 octobre. L'appel à sponsors est lancé : http://2012.pgconf.eu/

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA.

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

(lien vers l'article original)

Revues de code

Correctifs appliqués

Bruce Momjian a poussé :

Robert Haas a poussé :

Peter Eisentraut a poussé :

  • Mark ReThrowError() with attribute noreturn. All related functions were already so marked. http://git.postgresql.org/pg/commitdiff/26471a51fc833e2ce58a2f16f891256d57dd28c6
  • Improve markup of cmdsynopsis elements. Add more markup in particular so that the command options appear consistently in monospace in the HTML output. On the vacuumdb reference page, remove listing all the possible options in the synopsis. They have become too many now; we have the detailed options list for that. http://git.postgresql.org/pg/commitdiff/4266509c577b089627930af39f1dcd2d06b493e9
  • Fix display of <command> elements on man pages. We had changed this from the default bold to monospace for all output formats, but for man pages, this creates visual inconsistencies, so revert to the default for man pages. http://git.postgresql.org/pg/commitdiff/61c84b47619c11e74089cb3160813a4b3c98e6d7
  • Remove dead ports. Remove the following ports: dgux, nextstep, sunos4, svr4, ultrix4, and univel. These are obsolete and not worth rescuing. In most cases, there is circumstantial evidence that they wouldn't work anymore anyway. http://git.postgresql.org/pg/commitdiff/f2f9439fbfba378cb64cd6e5a046e0184cd542c6
  • Even more duplicate word removal, in the spirit of the season http://git.postgresql.org/pg/commitdiff/e9605a039b60350003daf8a5b3c0c10993994b60
  • PL/Python: Fix crash in functions returning SETOF and using SPI. Allocate PLyResultObject.tupdesc in TopMemoryContext, because its lifetime is the lifetime of the Python object and it shouldn't be freed by some other memory context, such as one controlled by SPI. We trust that the Python object will clean up its own memory. Before, this would crash the included regression test case by trying to use memory that was already freed. reported by Asif Naeem, analysis by Tom Lane http://git.postgresql.org/pg/commitdiff/52aa334fcd5a9d230be7e8fb964d94c6c4e63dc7
  • doc: Fix for too many brackets in command synopses on man pages. The default for the choice attribute of the <arg> element is "opt", which would normally put the argument inside brackets. But the DSSSL stylesheets contain a hack that treats <arg> directly inside <group> specially, so that <group><arg>-x</arg><arg>-y</arg></group> comes out as [ -x | -y ] rather than [ [-x] | [-y] ], which it would technically be. But when building man pages, this doesn't work, and so the command synopses on the man pages contain lots of extra brackets. By putting choice="opt" or choice="plain" explicitly on every <arg> and <group> element, we avoid any toolchain dependencies like that, and it also makes it clearer in the source code what is meant. In passing, make some small corrections in the documentation about which arguments are really optional or not. http://git.postgresql.org/pg/commitdiff/1715ff112809bca5218ddb6eccfda2c20dc420b5
  • PL/Python: Improve test coverage. Add test cases for inline handler of plython2u (when using that language name), and for result object element assignment. There is now at least one test case for every top-level functionality, except plpy.Fatal (annoying to use in regression tests) and result object slice retrieval and slice assignment (which are somewhat broken). http://git.postgresql.org/pg/commitdiff/e6c2e8cb87846161033e1f215876c4b95f631df0

Tom Lane a poussé :

  • Converge all SQL-level statistics timing values to float8 milliseconds. This patch adjusts the core statistics views to match the decision already taken for pg_stat_statements, that values representing elapsed time should be represented as float8 and measured in milliseconds. By using float8, we are no longer tied to a specific maximum precision of timing data. (Internally, it's still microseconds, but we could now change that without needing changes at the SQL level.) The columns affected are pg_stat_bgwriter.checkpoint_write_time, pg_stat_bgwriter.checkpoint_sync_time, pg_stat_database.blk_read_time, pg_stat_database.blk_write_time, pg_stat_user_functions.total_time, pg_stat_user_functions.self_time, pg_stat_xact_user_functions.total_time, and pg_stat_xact_user_functions.self_time. The first four of these are new in 9.2, so there is no compatibility issue from changing them. The others require a release note comment that they are now double precision (and can show a fractional part) rather than bigint as before; also their underlying statistics functions now match the column definitions, instead of returning bigint microseconds. http://git.postgresql.org/pg/commitdiff/809e7e21af8cd24855f1802524a13bbaa823f929
  • Kill some remaining references to SVR4 and univel. Both terms still appear in a few places, but I thought it best to leave those alone in context. http://git.postgresql.org/pg/commitdiff/50c2d6a1a63f04fd8c4553fc696c2c9e235b1a25
  • Overdue code review for transaction-level advisory locks patch. Commit 62c7bd31c8878dd45c9b9b2429ab7a12103f3590 had assorted problems, most visibly that it broke PREPARE TRANSACTION in the presence of session-level advisory locks (which should be ignored by PREPARE), as per a recent complaint from Stephen Rees. More abstractly, the patch made the LockMethodData.transactional flag not merely useless but outright dangerous, because in point of fact that flag no longer tells you anything at all about whether a lock is held transactionally. This fix therefore removes that flag altogether. We now rely entirely on the convention already in use in lock.c that transactional lock holds must be owned by some ResourceOwner, while session holds are never so owned. Setting the locallock struct's owner link to NULL thus denotes a session hold, and there is no redundant marker for that. PREPARE TRANSACTION now works again when there are session-level advisory locks, and it is also able to transfer transactional advisory locks to the prepared transaction, but for implementation reasons it throws an error if we hold both types of lock on a single lockable object. Perhaps it will be worth improving that someday. Assorted other minor cleanup and documentation editing, as well. Back-patch to 9.1, except that in the 9.1 branch I did not remove the LockMethodData.transactional flag for fear of causing an ABI break for any external code that might be examining those structs. http://git.postgresql.org/pg/commitdiff/71b9549d053b2f0a9e76e829c917385841f84bee

Heikki Linnakangas a poussé :

Magnus Hagander a poussé :

Correctifs rejetés (à ce jour)

  • Pas de déception cette semaine :-)

Correctifs en attente

  • Ryan Kelly sent in another revision of the patch to allow breaking out of hung connection attempts in libpq.
  • Noah Misch sent in a patch to prevent a theoretical torn page hazard in ginRedoUpdateMetapage().
  • Pavel Stehule sent in a patch to add new error fields to PL/pgsql.
  • Laurenz Albe sent in another revision of the patch to analyze foreign tables which gets the FDW to show that a value was non-NULL but removed due to excess width by returning a value of length WIDTH_THRESHOLD+1.
  • Peter Geoghegan sent in two revisions of a patch to latch up the WAL Writer, reducing wake-ups and thus saving electricity in a way that is more-or-less analogous to his previous work on the BGWriter.
  • Magnus Hagander sent in a patch to reduce the number of "Unexpected EOF on client connection" messages clogging people's logs.
  • Jan Urbanski sent in a patch to fix an issue with PL/PythonU where result set slicing was broken in the Python3 case.

par N Bougain le lundi 7 mai 2012 à 15h14

mardi 1 mai 2012

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 29 avril 2012

Offres d'emplois autour de PostgreSQL en avril

PostgreSQL Local

  • La PGCon 2012 sera tenue à l'Université d'Ottawa, les 17 et 18 mai 2012. Elle sera précédée par deux jours de tutoriels les 15 & 16 mai 2012 : http://www.pgcon.org/2012/
  • Le PGDay France aura lieu à Lyon, le 7 juin 2012 : http://www.pgday.fr
  • La PostgreSQL Conference Europe 2012 aura lieu à Prague, République Tchèque, du 23 au 26 octobre. L'appel à sponsors est lancé : http://2012.pgconf.eu/

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA.

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

(lien vers l'article original)

Revues de code

Correctifs appliqués

Robert Haas a poussé :

Peter Eisentraut a poussé :

Tom Lane a poussé :

  • Another trivial comment-typo fix. http://git.postgresql.org/pg/commitdiff/9873001e6d1177d543a2e46273e738d726670f1f
  • Fix edge-case behavior of pg_next_dst_boundary(). Due to rather sloppy thinking (on my part, I'm afraid) about the appropriate behavior for boundary conditions, pg_next_dst_boundary() gave undefined, platform-dependent results when the input time is exactly the last recorded DST transition time for the specified time zone, as a result of fetching values one past the end of its data arrays. Change its specification to be that it always finds the next DST boundary *after* the input time, and adjust code to match that. The sole existing caller, DetermineTimeZoneOffset, doesn't actually care about this distinction, since it always uses a probe time earlier than the instant that it does care about. So it seemed best to me to change the API to make the result=1 and result=0 cases more consistent, specifically to ensure that the "before" outputs always describe the state at the given time, rather than hacking the code to obey the previous API comment exactly. Per bug #6605 from Sergey Burladyan. Back-patch to all supported versions. http://git.postgresql.org/pg/commitdiff/c62b8eaae11aaa69a2b71bc63f9f78ca72eb412c
  • Fix planner's handling of RETURNING lists in writable CTEs. setrefs.c failed to do "rtoffset" adjustment of Vars in RETURNING lists, which meant they were left with the wrong varnos when the RETURNING list was in a subquery. That was never possible before writable CTEs, of course, but now it's broken. The executor fails to notice any problem because ExecEvalVar just references the ecxt_scantuple for any normal varno; but EXPLAIN breaks when the varno is wrong, as illustrated in a recent complaint from Bartosz Dmytrak. Since the eventual rtoffset of the subquery is not known at the time we are preparing its plan node, the previous scheme of executing set_returning_clause_references() at that time cannot handle this adjustment. Fortunately, it turns out that we don't really need to do it that way, because all the needed information is available during normal setrefs.c execution; we just have to dig it out of the ModifyTable node. So, do that, and get rid of the kluge of early setrefs processing of RETURNING lists. (This is a little bit of a cheat in the case of inherited UPDATE/DELETE, because we are not passing a "root" struct that corresponds exactly to what the subplan was built with. But that doesn't matter, and anyway this is less ugly than early setrefs processing was.) Back-patch to 9.1, where the problem became possible to hit. http://git.postgresql.org/pg/commitdiff/9fa82c980935ef4aee18fabe8da20ae2198b052a
  • Modify create_index regression test to avoid intermittent failures. We have been seeing intermittent buildfarm failures due to a query sometimes not using an index-only scan plan, because a background auto-ANALYZE prevented the table's all-visible bits from being set immediately, thereby causing the estimated cost of an index-only scan to go up considerably. Adjust the test case so that a bitmap index scan is preferred instead, which serves equally well for the purpose the test case is actually meant for. (Of course, it would be better to eliminate the interference from auto-ANALYZE, but I see no low-risk way to do that, so any such fix will have to be left for 9.3 or later.) http://git.postgresql.org/pg/commitdiff/d6d5f67b5b98b1685f9158e9d00a726afb2ae789
  • Fix oversight in recent parameterized-path patch. bitmap_scan_cost_est() has to be able to cope with a BitmapOrPath, but I'd taken a shortcut that didn't work for that case. Noted by Heikki. Add some regression tests since this area is evidently under-covered. http://git.postgresql.org/pg/commitdiff/7c85aa39fc08df44e1ce67e651bda4cf7e331580
  • Improve documentation around historical calendar rules. Get rid of section 8.5.6 (Date/Time Internals), which appears to confuse people more than it helps, and anyway discussion of Postgres' internal datetime calculation methods seems pretty out of place here. Instead, make datatype.sgml just say that we follow the Gregorian calendar (a bit of specification not previously present anywhere in that chapter :-() and link to the History of Units appendix for more info. Do some mild editorialization on that appendix, too, to make it clearer that we are following proleptic Gregorian calendar rules rather than anything more historically accurate. Per a question from Florence Cousin and subsequent discussion in pgsql-docs. http://git.postgresql.org/pg/commitdiff/92df2203437603d40417fe711c3cb7066ac4fdf5
  • Fix syslogger's rotation disable/re-enable logic. If it fails to open a new log file, the syslogger assumes there's something wrong with its parameters (such as log_directory), and stops attempting automatic time-based or size-based log file rotations. Sending it SIGHUP is supposed to start that up again. However, the original coding for that was really bogus, involving clobbering a couple of GUC variables and hoping that SIGHUP processing would restore them. Get rid of that technique in favor of maintaining a separate flag showing we've turned rotation off. Per report from Mark Kirkwood. Also, the syslogger will automatically attempt to create the log_directory directory if it doesn't exist, but that was only happening at startup. For consistency and ease of use, it should do the same whenever the value of log_directory is changed by SIGHUP. Back-patch to all supported branches. http://git.postgresql.org/pg/commitdiff/537b26695389ae67adc4fbbe04254bc527a11128
  • Fix printing of whole-row Vars at top level of a SELECT targetlist. Normally whole-row Vars are printed as "tabname.*". However, that does not work at top level of a targetlist, because per SQL standard the parser will think that the "*" should result in column-by-column expansion; which is not at all what a whole-row Var implies. We used to just print the table name in such cases, which works most of the time; but it fails if the table name matches a column name available anywhere in the FROM clause. This could lead for instance to a view being interpreted differently after dump and reload. Adding parentheses doesn't fix it, but there is a reasonably simple kluge we can use instead: attach a no-op cast, so that the "*" isn't syntactically at top level anymore. This makes the printing of such whole-row Vars a lot more consistent with other Vars, and may indeed fix more cases than just the reported one; I'm suspicious that cases involving schema qualification probably didn't work properly before, either. Per bug report and fix proposal from Abbas Butt, though this patch is quite different in detail from his. Back-patch to all supported versions. http://git.postgresql.org/pg/commitdiff/d6f7d4fdc516b2b597f8c2cd011c41c2729dab45
  • Clear I/O timing counters after sending them to the stats collector. This oversight caused the reported times to accumulate in an O(N2) fashion the longer a backend runs. http://git.postgresql.org/pg/commitdiff/cdbad241f41362aaf09f913722a541e04e048742
  • Adjust timing units in pg_stat_statements. Display total time and I/O timings in milliseconds, for consistency with the units used for timings in the core statistics views. The columns remain of float8 type, so that sub-msec precision is available. (At some point we will probably want to convert the core views to use float8 type for the same reason, but this patch does not touch that issue.) This is a release-note-requiring change in the meaning of the total_time column. The I/O timing columns are new as of 9.2, so there is no compatibility impact from redefining them. Do some minor copy-editing in the documentation, too. http://git.postgresql.org/pg/commitdiff/93f94e356d47ea20ca7c2fcb65cbb746049fe4d1
  • Make a copy-editing pass over the new documentation for statistics views. Fix a bunch of typos, improve markup, make wording more uniform, rearrange some material. No substantive changes. http://git.postgresql.org/pg/commitdiff/aebe989477ac5a9f7b59ae464ec68ec45975ed3f
  • Further editorialization on the new documentation for statistics views. Get rid of the per-column documentation of underlying functions, which did far more to clutter the view descriptions than it did to be helpful, and was rather incomplete and typo-ridden anyway. Instead suggest that people consult the definitions of the standard views to see the underlying functions. The older functions for obtaining individual facts about backends are now somewhat obsoleted by pg_stat_get_activity, which means that they are not documented by any standard view. So I put that information into a separate table. (Maybe we should just deprecate them instead?) In passing, fix a couple more documentation errors. http://git.postgresql.org/pg/commitdiff/5f2b0893871cce1ffb77ac7f13b3fba227e6f11f
  • Rename track_iotiming GUC to track_io_timing. This spelling seems significantly more readable to me. http://git.postgresql.org/pg/commitdiff/309c64745ea145d7c731e1fe610631b2b84e7e88
  • Rename I/O timing statistics columns to blk_read_time and blk_write_time. This seems more consistent with the pre-existing choices for names of other statistics columns. Rename assorted internal identifiers to match. http://git.postgresql.org/pg/commitdiff/1dd89eadcd2648d7ca0baed3c7af16a04eb1aa26

Bruce Momjian a poussé :

Correctifs rejetés (à ce jour)

  • Pas de déception cette semaine :-)

Correctifs en attente

  • Zoltan Boszormenyi sent in another version of the patches to create a lock timeout framework and use same.
  • Alexander Korotkov sent in a patch to convert from pg_wchar to multibyte, and Erik Rijkers followed up with some regression tests for same.
  • KaiGai Kohei sent in another revision of the patch to add generic extra daemons.
  • Noah Misch sent in a patch to fix an issue where psql does not include a row count when auto-expanded output (\x auto) is set.
  • Kevin Grittner sent in a patch to fix an issue where setting defautl_transaction_isolation to serializable (SSI) broke Hot Standbys.
  • Robert Haas sent in a patch to modify "smart" shutdown mode to disconnect sessions that are not in a transaction (or as soon as they no longer are) but leaves in-progress transactions alone;
  • Noah Misch sent in a patch to allow TEMP tables on a hot standby.

par N Bougain le mardi 1 mai 2012 à 17h44

lundi 23 avril 2012

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 22 avril 2012

La PostgreSQL Conference Europe 2012 aura lieu à Prague, République Tchèque, du 23 au 26 octobre. L'appel à sponsors est ouvert : http://2012.pgconf.eu/

PGNext a été annulé.

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en avril

PostgreSQL Local

  • La seconde réunion du PUG de l'Arizona est programmée pour le mercredi 25 avril à 18h30 et débutera avec pizzas et rafraichissements. Vous pouvez rejoindre la mailing-list, et RSVP à : https://www.bigtent.com/groups/azpug
  • La PGCon 2012 sera tenue à l'Université d'Ottawa, les 17 et 18 mai 2012. Elle sera précédée par deux jours de tutoriels les 15 & 16 mai 2012 : http://www.pgcon.org/2012/
  • Le PGDay France aura lieu à Lyon, le 7 juin 2012 : http://www.pgday.fr

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA.

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

(lien vers l'article original)

Revues de code

Correctifs appliqués

Peter Eisentraut a poussé :

Heikki Linnakangas a poussé :

  • Install plpgsql.h to to include/server at "make install". The header file is needed by any module that wants to use the PL/pgSQL instrumentation plugin interface. Most notably, the pldebugger plugin needs this. With this patch, it can be built using pgxs, without having the full server source tree available. http://git.postgresql.org/pg/commitdiff/49440fff08590cf1a764ab4b4c5342cd9e445991
  • Don't wait for the commit record to be replicated if we wrote no WAL. When using synchronous replication, we waited for the commit record to be replicated, but if we our transaction didn't write any other WAL records, that's not required because we don't even flush the WAL locally to disk in that case. This lead to long waits when committing a transaction that only modified a temporary table. Bug spotted by Thom Brown. http://git.postgresql.org/pg/commitdiff/fe546f3da6a5ff1d879f587728f74ec457f0ee5f

Andrew Dunstan a poussé :

  • Don't override arguments set via options with positional arguments. A number of utility programs were rather careless about paremeters that can be set via both an option argument and a positional argument. This leads to results which can violate the Principal Of Least Astonishment. These changes refuse to use positional arguments to override settings that have been made via positional arguments. The changes are backpatched to all live branches. http://git.postgresql.org/pg/commitdiff/1b37a8c3cc4f0615f80d6007e2bbd47c6bd7e1e3

Robert Haas a poussé :

Tom Lane a poussé :

  • Revise parameterized-path mechanism to fix assorted issues. This patch adjusts the treatment of parameterized paths so that all paths with the same parameterization (same set of required outer rels) for the same relation will have the same rowcount estimate. We cache the rowcount estimates to ensure that property, and hopefully save a few cycles too. Doing this makes it practical for add_path_precheck to operate without a rowcount estimate: it need only assume that paths with different parameterizations never dominate each other, which is close enough to true anyway for coarse filtering, because normally a more-parameterized path should yield fewer rows thanks to having more join clauses to apply. In add_path, we do the full nine yards of comparing rowcount estimates along with everything else, so that we can discard parameterized paths that don't actually have an advantage. This fixes some issues I'd found with add_path rejecting parameterized paths on the grounds that they were more expensive than not-parameterized ones, even though they yielded many fewer rows and hence would be cheaper once subsequent joining was considered. To make the same-rowcounts assumption valid, we have to require that any parameterized path enforce *all* join clauses that could be obtained from the particular set of outer rels, even if not all of them are useful for indexing. This is required at both base scans and joins. It's a good thing anyway since the net impact is that join quals are checked at the lowest practical level in the join tree. Hence, discard the original rather ad-hoc mechanism for choosing parameterization joinquals, and build a better one that has a more principled rule for when clauses can be moved. The original rule was actually buggy anyway for lack of knowledge about which relations are part of an outer join's outer side; getting this right requires adding an outer_relids field to RestrictInfo. http://git.postgresql.org/pg/commitdiff/5b7b5518d0ea56c422a197875f7efa5deddbb388
  • Adjust join_search_one_level's handling of clauseless joins. For an initial relation that lacks any join clauses (that is, it has to be cartesian-product-joined to the rest of the query), we considered only cartesian joins with initial rels appearing later in the initial-relations list. This creates an undesirable dependency on FROM-list order. We would never fail to find a plan, but perhaps we might not find the best available plan. Noted while discussing the logic with Amit Kapila. Improve the comments a bit in this area, too. Arguably this is a bug fix, but given the lack of complaints from the field I'll refrain from back-patching. http://git.postgresql.org/pg/commitdiff/1f0363001166ef6a43619846e44cfb9dbe7335ed
  • Use fuzzy not exact cost comparison for the final tie-breaker in add_path. Instead of an exact cost comparison, use a fuzzy comparison with 1e-10 delta after all other path metrics have proved equal. This is to avoid having platform-specific roundoff behaviors determine the choice when two paths are really the same to our cost estimators. Adjust the recently-added test case that made it obvious we had a problem here. http://git.postgresql.org/pg/commitdiff/33e99153e93b9accfa51ac036828144e1c2507b7

Alvaro Herrera a poussé :

  • Recast "ONLY" column CHECK constraints as NO INHERIT. The original syntax wasn't universally loved, and it didn't allow its usage in CREATE TABLE, only ALTER TABLE. It now works everywhere, and it also allows using ALTER TABLE ONLY to add an uninherited CHECK constraint, per discussion. The pg_constraint column has accordingly been renamed connoinherit. This commit partly reverts some of the changes in 61d81bd28dbec65a6b144e0cd3d0bfe25913c3ac, particularly some pg_dump and psql bits, because now pg_get_constraintdef includes the necessary NO INHERIT within the constraint definition. Author: Nikhil Sontakke. Some tweaks by me http://git.postgresql.org/pg/commitdiff/09ff76fcdb275769ac4d1a45a67416735613d04b

Correctifs rejetés (à ce jour)

  • Pas de déception cette semaine :-)

Correctifs en attente

  • Heikki Linnakangas sent in a patch to fix some infelicities in localized error context messages.
  • Etsuro Fujita sent in another revision of the patch to validate file_fdw tables for not-NULL constraints.
  • Kyotaro HORIGUCHI sent in three revisions of a fix for a bug where the checkpointer on hot standbys runs without looking checkpoint_segments. Fujii Masao sent in another fixing an infelicity in the last.
  • Noah Misch sent in a patch to fix an issue where {ts,array}_typanalyze consumed much more memory than needed.
  • Jameison Martin sent in a patch to optimize the case of tables with large numbers of columns, truncating the NULL bitmap at the last non-NULL column.
  • Alexander Shulgin sent in two more revisions of a patch to add libpq URIs to the regression tests.
  • Alvaro Herrera sent in a patch to add ALTER EXTENSION ... OWNED.
  • Robert Haas sent in a patch to fix a bug in lazy_scan_heap() that could cause index-only scans to give wrong results.
  • Laurenz Albe sent in a patch to fix a place where foreign table scan estimates were frozen at 1000 rows, independent of statistics gathered on same.
  • Noah Misch sent in a patch to fix an issue in B-tree page deletion.
  • Jan Urbanski sent in a patch to fix an issue with PL/PythonU triggers on composite-type columns. columns

par N Bougain le lundi 23 avril 2012 à 23h13

jeudi 19 avril 2012

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 1er avril 2012

Au terme d'une campagne éclair, Oracle a pris le contrôle de toutes les entreprises offrant un support de PostgreSQL : Bull, Command Prompt, Dalibo, EMC, EnterpriseDB, OmniTI, PostgreSQL Experts, Second Quadrant et VMware. Larry Ellison, PDG, a déclaré que ces aquisitions étaient nécessaires à la construction de son prochain yacht, lui permettant d'être encore plus grand que le précédent.

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en avril

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA.

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

(lien vers l'article original)

Revues de code

Correctifs appliqués

Robert Haas a poussé :

Peter Eisentraut a poussé :

Tom Lane a poussé :

  • Silence compiler warning about uninitialized variable. http://git.postgresql.org/pg/commitdiff/98316e211b60cb160247171e3557b40a247c4610
  • Bend parse location rules for the convenience of pg_stat_statements. Generally, the parse location assigned to a multiple-token construct is the location of its leftmost token. This commit breaks that rule for the syntaxes TYPENAME 'LITERAL' and CAST(CONSTANT Alexander Shulgin TYPENAME) --- the resulting Const will have the location of the literal string, not the typename or CAST keyword. The cases where this matters are pretty thin on the ground (no error messages in the regression tests change, for example), and it's unlikely that any user would be confused anyway by an error cursor pointing at the literal. But still it's less than consistent. The reason for changing it is that contrib/pg_stat_statements wants to know the parse location of the original literal, and it was agreed that this is the least unpleasant way to preserve that information through parse analysis. Peter Geoghegan http://git.postgresql.org/pg/commitdiff/5d3fcc4c2e137417ef470d604fee5e452b22f6a7
  • Add some infrastructure for contrib/pg_stat_statements. Add a queryId field to Query and PlannedStmt. This is not used by the core backend, except for being copied around at appropriate times. It's meant to allow plug-ins to track a particular query forward from parse analysis to execution. The queryId is intentionally not dumped into stored rules (and hence this commit doesn't bump catversion). You could argue that choice either way, but it seems better that stored rule strings not have any dependency on plug-ins that might or might not be present. Also, add a post_parse_analyze_hook that gets invoked at the end of parse analysis (but only for top-level analysis of complete queries, not cases such as analyzing a domain's default-value expression). This is mainly meant to be used to compute and assign a queryId, but it could have other applications. Peter Geoghegan http://git.postgresql.org/pg/commitdiff/a40fa613b516b97c37d87ac1b21fb7aa8a2f2c1b
  • Improve contrib/pg_stat_statements to lump "similar" queries together. pg_stat_statements now hashes selected fields of the analyzed parse tree to assign a "fingerprint" to each query, and groups all queries with the same fingerprint into a single entry in the pg_stat_statements view. In practice it is expected that queries with the same fingerprint will be equivalent except for values of literal constants. To make the display more useful, such constants are replaced by "?" in the displayed query strings. This mechanism currently supports only optimizable queries (SELECT, INSERT, UPDATE, DELETE). Utility commands are still matched on the basis of their literal query strings. There remain some open questions about how to deal with utility statements that contain optimizable queries (such as EXPLAIN and SELECT INTO) and how to deal with expiring speculative hashtable entries that are made to save the normalized form of a query string. However, fixing these issues should require only localized changes, and since there are other open patches involving contrib/pg_stat_statements, it seems best to go ahead and commit what we've got. Peter Geoghegan, reviewed by Daniel Farina http://git.postgresql.org/pg/commitdiff/7313cc016344a5705eb3e6916d8c4ea849c57975
  • Improve handling of utility statements containing plannable statements. When tracking nested statements, contrib/pg_stat_statements formerly double-counted the execution costs of utility statements that directly contain an executable statement, such as EXPLAIN and DECLARE CURSOR. This was not obvious since the ProcessUtility and Executor hooks would each add their measured costs to the same stats table entry. However, with the new implementation that hashes utility and plannable statements differently, this showed up as seemingly-duplicate stats entries. Fix that by disabling the Executor hooks when the query has a queryId of zero, which was the case already for such statements but is now more clearly specified in the code. (The zero queryId was causing problems anyway because all such statements would add to a single bogus entry.) The PREPARE/EXECUTE case still results in counting the same execution in two different stats table entries, but it should be much less surprising to users that there are two entries in such cases. In passing, include a CommonTableExpr's ctename in the query hash. I had left it out originally on the grounds that we wanted to omit all inessential aliases, but since RTE_CTE RTEs are hashing their referenced names, we'd better hash the CTE names too to make sure we don't hash semantically different queries the same. http://git.postgresql.org/pg/commitdiff/e0e4ebe38469a777e3c585e1d97383f974c19b8c
  • Improve contrib/pg_stat_statements' handling of PREPARE/EXECUTE statements. It's actually more useful for the module to ignore these. Ignoring EXECUTE (and not incrementing the nesting level) allows the executor hooks to charge the time to the underlying prepared query, which shows up as a stats entry with the original PREPARE as query string (possibly modified by suppression of constants, which might not be terribly useful here but it's not worth avoiding). This is much more useful than cluttering the stats table with a distinct entry for each textually distinct EXECUTE. Experimentation with this idea shows that it's also preferable to ignore PREPARE. If we don't, we get two stats table entries, one with the query string hash and one with the jumble-derived hash, but with the same visible query string (modulo those constants). This is confusing and not very helpful, since the first entry will only receive costs associated with initial planning of the query, which is not something counted at all normally by pg_stat_statements. (And if we do start tracking planning costs, we'd want them blamed on the other hash table entry anyway.) http://git.postgresql.org/pg/commitdiff/566a1d43cf6bfcc7f9385b581d98e07eab282cdd
  • Fix dblink's failure to report correct connection name in error messages. The DBLINK_GET_CONN and DBLINK_GET_NAMED_CONN macros did not set the surrounding function's conname variable, causing errors to be incorrectly reported as having occurred on the "unnamed" connection in some cases. This bug was actually visible in two cases in the regression tests, but apparently whoever added those cases wasn't paying attention. Noted by Kyotaro Horiguchi, though this is different from his proposed patch. Back-patch to 8.4; 8.3 does not have the same type of error reporting so the patch is not relevant. http://git.postgresql.org/pg/commitdiff/b75fbe91910df323a8d3e1d92a8bb4dd0d5e88a9
  • Add PGDLLIMPORT to ScanKeywords and NumScanKeywords. Per buildfarm, this is now needed by contrib/pg_stat_statements. http://git.postgresql.org/pg/commitdiff/5e83854d71bb05403768a97a415a129b0081564b
  • Fix glitch recently introduced in psql tab completion. Over-optimization (by me, looks like :-() broke the case of recognizing a word boundary just before a quoted identifier. Reported and diagnosed by Dean Rasheed. http://git.postgresql.org/pg/commitdiff/a52e6fe7bcf86f7e52d7b1d6f59260cb57b565fa
  • Rename frontend keyword arrays to avoid conflict with backend. ecpg and pg_dump each contain keyword arrays with structure similar to the backend's keyword array. Up to now, we actually named those arrays the same as the backend's and relied on parser/keywords.h to declare them. This seems a tad too cute, though, and it breaks now that we need to PGDLLIMPORT-decorate the backend symbols. Rename to avoid the problem. Per buildfarm. (It strikes me that maybe we should get rid of the separate keywords.c files altogether, and just define these arrays in the modules that use them, but that's a rather more invasive change.) http://git.postgresql.org/pg/commitdiff/c252a17d828756e2f7d635f69eace53aaf983420
  • Fix O(N2) behavior in pg_dump for large numbers of owned sequences. The loop that matched owned sequences to their owning tables required time proportional to number of owned sequences times number of tables; although this work was only expended in selective-dump situations, which is probably why the issue wasn't recognized long since. Refactor slightly so that we can perform this work after the index array for findTableByOid has been set up, reducing the time to O(M log N). Per gripe from Mike Roest. Since this is a longstanding performance bug, backpatch to all supported versions. http://git.postgresql.org/pg/commitdiff/0d8117abefdae69dbec7465adf2c68f5cd0412ac
  • Fix O(N2) behavior in pg_dump when many objects are in dependency loops. Combining the loop workspace with the record of already-processed objects might have been a cute trick, but it behaves horridly if there are many dependency loops to repair: the time spent in the first step of findLoop() grows as O(N2). Instead use a separate flag array indexed by dump ID, which we can check in constant time. The length of the workspace array is now never more than the actual length of a dependency chain, which should be reasonably short in all cases of practical interest. The code is noticeably easier to understand this way, too. Per gripe from Mike Roest. Since this is a longstanding performance bug, backpatch to all supported versions. http://git.postgresql.org/pg/commitdiff/d5881c036a913d31a5b0f56519cce76ca3b3e587

Andrew Dunstan a poussé :

Heikki Linnakangas a poussé :

  • Inherit max_safe_fds to child processes in EXEC_BACKEND mode. Postmaster sets max_safe_fds by testing how many open file descriptors it can open, and that is normally inherited by all child processes at fork(). Not so on EXEC_BACKEND, ie. Windows, however. Because of that, we effectively ignored max_files_per_process on Windows, and always assumed a conservative default of 32 simultaneous open files. That could have an impact on performance, if you need to access a lot of different files in a query. After this patch, the value is passed to child processes by save/restore_backend_variables() among many other global variables. It has been like this forever, but given the lack of complaints about it, I'm not backpatching this. http://git.postgresql.org/pg/commitdiff/5762a4d9098ac0cba789ddd26286ac85c2d316f2

Simon Riggs a poussé :

  • Correct epoch of txid_current() when executed on a Hot Standby server. Initialise ckptXidEpoch from starting checkpoint and maintain the correct value as we roll forwards. This allows GetNextXidAndEpoch() to return the correct epoch when executed during recovery. Backpatch to 9.0 when the problem is first observable by a user. Bug report from Daniel Farina http://git.postgresql.org/pg/commitdiff/68219aaf6b8c6214e5f4a6124bd5607254aad6b3

Correctifs rejetés (à ce jour)

  • Pas de déception cette semaine :-)

Correctifs en attente

  • Kyotaro HORIGUCHI sent in two more revisions of the patch to create a new tuple storage format for libpq and use same in dblink.
  • Shigeru HANADA sent in two more revisions of the patch to add a PostgreSQL FDW along with infrastructure for same.
  • Peter Eisentraut and Alexander Shulgin traded patches to add a URI format for connection strings in libpq.
  • Fujii Masao sent in two revisions of a patch to make pg_basebackup exit on error.
  • Ants Aasma sent in a patch to use lazy hash aggregation to speed up cases where no actual aggregates are used.
  • Dimitri Fontaine sent in two more revisions of the patch to add finer dependencies for EXTENSIONs.
  • Marco Nenciarini sent in another revision of the patch to allow each element of an array to be an enforced foreign key reference.
  • Peter Eisentraut sent in a patch to fix some infelicities between pgxs, bison and flexx.
  • Andrew Dunstan and Joachim Wieland traded patches to implement parallel pg_dump.
  • Zoltan Boszormenyi sent in two more revisions of the ECPG FETCH readahead patch.
  • Daniel Farina sent in another revision of the patch to allow same-role pg_terminate_backend.
  • Pavel Stehule sent in another revision of the CHECK TRIGGER functionality for PL/pgsql.
  • Peter Eisentraut sent in a patch which reverts the default capitalization behavior in psql's tab completion to that prior to a previous patch while expanding the tunability of that capitalization with tab completion.
  • Robert Haas sent in two patches to measure lwlock-related latency spikes.
  • Heikki Linnakangas sent in a patch to set stack_base_ptr in autovacuum.

par N Bougain le jeudi 19 avril 2012 à 23h29

Nouvelles hebdomadaires de PostgreSQL - 15 avril 2012

La seconde réunion du PUG de l'Arizona est programmée pour le mercredi 25 avril à 18h30 et débutera avec pizzas et rafraichissements. Vous pouvez rejoindre la mailing-list, et RSVP à : https://www.bigtent.com/groups/azpug

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en avril

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA.

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

(lien vers l'article original)

Revues de code

Correctifs appliqués

Heikki Linnakangas a poussé :

Tom Lane a poussé :

  • Save a few cycles while creating "sticky" entries in pg_stat_statements. There's no need to sit there and increment the stats when we know all the increments would be zero anyway. The actual additions might not be very expensive, but skipping acquisition of the spinlock seems like a good thing. Pushing the logic about initialization of the usage count down into entry_alloc() allows us to do that while making the code actually simpler, not more complex. Expansion on a suggestion by Peter Geoghegan. http://git.postgresql.org/pg/commitdiff/e969f9a78008d6a09abf8646f1338e2dff447cbf
  • Don't bother copying empty support arrays in a zero-column MergeJoin. The case could not arise when this code was originally written, but it can now (since we made zero-column MergeJoins work for the benefit of FULL JOIN ON TRUE). I don't think there is any actual bug here, but we might as well treat it consistently with other uses of COPY_POINTER_FIELD(). Per comment from Ashutosh Bapat. http://git.postgresql.org/pg/commitdiff/d515365a611a58241019c59a62b0cb79584aa725
  • Fix an Assert that turns out to be reachable after all. estimate_num_groups() gets unhappy with create table empty(); select * from empty except select * from empty e2; I can't see any actual use-case for such a query (and the table is illegal per SQL spec), but it seems like a good idea that it not cause an assert failure. http://git.postgresql.org/pg/commitdiff/65fd91333e45114c5d9a07d3d4f6a4786df48768
  • Adjust various references to GEQO being non-deterministic. It's still non-deterministic in some sense ... but given fixed settings and identical planning problems, it will now always choose the same plan, so we probably shouldn't tar it with that brush. Per bug #6565 from Guillaume Cottenceau. Back-patch to 9.0 where the behavior was fixed. http://git.postgresql.org/pg/commitdiff/c94b43ce39f3ee48db04fb8284de0db12526af92
  • Measure epoch of timestamp-without-time-zone from local not UTC midnight. This patch reverts commit 191ef2b407f065544ceed5700e42400857d9270f and thereby restores the pre-7.3 behavior of EXTRACT(EPOCH FROM timestamp-without-tz). Per discussion, the more recent behavior was misguided on a couple of grounds: it makes it hard to get a non-timezone-aware epoch value for a timestamp, and it makes this one case dependent on the value of the timezone GUC, which is incompatible with having timestamp_part() labeled as immutable. The other behavior is still available (in all releases) by explicitly casting the timestamp to timestamp with time zone before applying EXTRACT. This will need to be called out as an incompatible change in the 9.2 release notes. Although having mutable behavior in a function marked immutable is clearly a bug, we're not going to back-patch such a change. http://git.postgresql.org/pg/commitdiff/0d9819f7e3be3c4186f0398d007d2fef8809da6a
  • Make pg_tablespace_location(0) return the database's default tablespace. This definition is convenient when applying the function to the reltablespace column of pg_class, since that's what zero means there; and it doesn't interfere with any other plausible use of the function. Per gripe from Bruce Momjian. http://git.postgresql.org/pg/commitdiff/3769fa5fc64298143cf535c8a06921793598e458
  • Silently ignore any nonexistent schemas that are listed in search_path. Previously we attempted to throw an error or at least warning for missing schemas, but this was done inconsistently because of implementation restrictions (in many cases, GUC settings are applied outside transactions so that we can't do system catalog lookups). Furthermore, there were exceptions to the rule even in the beginning, and we'd been poking more and more holes in it as time went on, because it turns out that there are lots of use-cases for having some irrelevant items in a common search_path value. It seems better to just adopt a philosophy similar to what's always been done with Unix PATH settings, wherein nonexistent or unreadable directories are silently ignored. This commit also fixes the documentation to point out that schemas for which the user lacks USAGE privilege are silently ignored. That's always been true but was previously not documented. This is mostly in response to Robert Haas' complaint that 9.1 started to throw errors or warnings for missing schemas in cases where prior releases had not. We won't adopt such a significant behavioral change in a back branch, so something different will be needed in 9.1. http://git.postgresql.org/pg/commitdiff/880bfc3287dd68cfe90d10d9597d7b0fd2dae3e5
  • Fix cost estimation for indexscan filter conditions. cost_index's method for estimating per-tuple costs of evaluating filter conditions (a/k/a qpquals) was completely wrong in the presence of derived indexable conditions, such as range conditions derived from a LIKE clause. This was largely masked in common cases as a result of all simple operator clauses having about the same costs, but it could show up in a big way when dealing with functional indexes containing expensive functions, as seen for example in bug #6579 from Istvan Endredy. Rejigger the calculation to give sane answers when the indexquals aren't a subset of the baserestrictinfo list. As a side benefit, we now do the calculation properly for cases involving join clauses (ie, parameterized indexscans), which we always overestimated before. There are still cases where this is an oversimplification, such as clauses that can be dropped because they are implied by a partial index's predicate. But we've never accounted for that in cost estimates before, and I'm not convinced it's worth the cycles to try to do so. http://git.postgresql.org/pg/commitdiff/732bfa2448c77bba5d98f485a387da77e1df8395
  • Assorted spelling corrections. Thom Brown http://git.postgresql.org/pg/commitdiff/81e3e4fd14f82e93edfda8621727040ab3c70840
  • Remove the "last ditch" code path in join_search_one_level(). So far as I can tell, it is no longer possible for this heuristic to do anything useful, because the new weaker definition of have_relevant_joinclause means that any relation with a joinclause must be considered joinable to at least one other relation. It would still be possible for the code block to be entered, for example if there are join order restrictions that prevent any join of the current level from being formed; but in that case it's just a waste of cycles to attempt to form cartesian joins, since the restrictions will still apply. Furthermore, IMO the existence of this code path can mask bugs elsewhere; we would have noticed the problem with cartesian joins a lot sooner if this code hadn't compensated for it in the simplest case. Accordingly, let's remove it and see what happens. I'm committing this separately from the prerequisite changes in have_relevant_joinclause, just to make the question easier to revisit if there is some fault in my logic. http://git.postgresql.org/pg/commitdiff/e54b10a62db2991235fe800c629baef4531a6d67
  • Weaken the planner's tests for relevant joinclauses. We should be willing to cross-join two small relations if that allows us to use an inner indexscan on a large relation (that is, the potential indexqual for the large table requires both smaller relations). This worked in simple cases but fell apart as soon as there was a join clause to a fourth relation, because the existence of any two-relation join clause caused the planner to not consider clauseless joins between other base relations. The added regression test shows an example case adapted from a recent complaint from Benoit Delbosc. Adjust have_relevant_joinclause, have_relevant_eclass_joinclause, and has_relevant_eclass_joinclause to consider that a join clause mentioning three or more relations is sufficient grounds for joining any subset of those relations, even if we have to do so via a cartesian join. Since such clauses are relatively uncommon, this shouldn't affect planning speed on typical queries; in fact it should help a bit, because the latter two functions in particular get significantly simpler. Although this is arguably a bug fix, I'm not going to risk back-patching it, since it might have currently-unforeseen consequences. http://git.postgresql.org/pg/commitdiff/e3ffd05b02468b1a53de31a322cedf195576a625

Bruce Momjian a poussé :

Peter Eisentraut a poussé :

Alvaro Herrera a poussé :

  • Accept postgres:// URIs in libpq connection functions. postgres:// URIs are an attempt to "stop the bleeding" in this general area that has been said to occur due to external projects adopting their own syntaxes. The syntaxes supported by this patch: postgres://[user[:pwd]@][unix-socket][:port[/dbname]][?param1=value1&...] postgres://[user[:pwd]@][net-location][:port][/dbname][?param1=value1&...] should be enough to cover most interesting cases without having to resort to "param=value" pairs, but those are provided for the cases that need them regardless. libpq documentation has been shuffled around a bit, to avoid stuffing all the format details into the PQconnectdbParams description, which was already a bit overwhelming. The list of keywords has moved to its own subsection, and the details on the URI format live in another subsection. This includes a simple test program, as requested in discussion, to ensure that interesting corner cases continue to work appropriately in the future. Author: Alexander Shulgin Some tweaking by Álvaro Herrera, Greg Smith, Daniel Farina, Peter Eisentraut Reviewed by Robert Haas, Alexey Klyukin (offlist), Heikki Linnakangas,Marko Kreen, and others Oh, it also supports postgresql:// but that's probably just an accident. http://git.postgresql.org/pg/commitdiff/b035cb9db7aa7c0f28581b23feb10d3c559701f6

Robert Haas a poussé :

Tatsuo Ishii a poussé :

Correctifs rejetés (à ce jour)

  • Etsuro Fujita's patch to fix a typo in a comment.

Correctifs en attente

  • Vikash S. sent in a patch intended to allow psql to take multiple -f (file) arguments on the command line.
  • Robert Haas sent in another revision of the patch implementing pg_prewarm.
  • Robert Haas sent in another revision of a patch to fix a bug in fastpath locking.
  • Zoltan Boszormenyi sent in another revision of the patch for ECPG FETCH readahead.
  • Josh Kupershmidt sent in another revision of the patch to fix psql tab completion in the WITH case of CREATE/ALTER USER/ROLE.
  • Zoltan Boszormenyi sent in new versions of two patches: timeout framework and lock timeout.
  • Nikhil Sontakke sent in a patch implementing CHECK ONLY in CREATE/ALTER TABLE.
  • Heikki Linnakangas sent in a patch to allow compiling PL/pgsql-related items as EXTENSIONs.
  • Robert Haas sent in several patches to test different scenarios for memory usage during sorting.
  • Etsuro Fujita sent in a patch to add some validating machinery to FDWs and used same to check NULLness in the file_fdw.
  • Heikki Linnakangas sent in a patch to ensure that the right domain is used in errcontext() calls.
  • Tatsuo Ishii sent in a patch ensuring that pg_ctl --help actually documents all options the code makes available.

par N Bougain le jeudi 19 avril 2012 à 23h27

Nouvelles hebdomadaires de PostgreSQL - 8 avril 2012

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en Avril

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA.

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

(lien vers l'article original)

Correctifs appliqués

Peter Eisentraut a poussé :

Robert Haas a poussé :

Tom Lane a poussé :

  • Fix a couple of contrib/dblink bugs. dblink_exec leaked temporary database connections if any error occurred after connection setup, for example SELECT dblink_exec('...connect string...', 'select 1/0'); Add a PG_TRY block to ensure PQfinish gets done when it is needed. (dblink_record_internal is on the hairy edge of needing similar treatment, but seems not to be actively broken at the moment.) Also, in 9.0 and up, only one of the three functions using tuplestore return mode was properly checking that the query context would allow a tuplestore result. Noted while reviewing dblink patch. Back-patch to all supported branches. http://git.postgresql.org/pg/commitdiff/d843ed21164275dfbfefa486bb00bca91380cdf0
  • Fix syslogger to not lose log coherency under high load. The original coding of the syslogger had an arbitrary limit of 20 large messages concurrently in progress, after which it would just punt and dump message fragments to the output file separately. Our ambitions are a bit higher than that now, so allow the data structure to expand as necessary. Reported and patched by Andrew Dunstan; some editing by Tom http://git.postgresql.org/pg/commitdiff/c17e863bc7677a54d6da5bbb2868cca2cd9b30c1
  • Remove useless PGRES_COPY_BOTH "support" in psql. There is no existing or foreseeable case in which psql should see a PGRES_COPY_BOTH PQresultStatus; and if such a case ever emerges, it's a pretty good bet that these code fragments wouldn't do the right thing anyway. Remove them, and let the existing default cases do the appropriate thing, namely emit an "unexpected PQresultStatus" bleat. Noted while working on libpq row processor patch, for which I was considering adding a PGRES_SUSPENDED status code --- the same default-case treatment would be appropriate for that. http://git.postgresql.org/pg/commitdiff/cb917e1544612c187c74fed1a990e26820514c8a
  • Add a "row processor" API to libpq for better handling of large results. Traditionally libpq has collected an entire query result before passing it back to the application. That provides a simple and transactional API, but it's pretty inefficient for large result sets. This patch allows the application to process each row on-the-fly instead of accumulating the rows into the PGresult. Error recovery becomes a bit more complex, but often that tradeoff is well worth making. Kyotaro Horiguchi, reviewed by Marko Kreen and Tom Lane http://git.postgresql.org/pg/commitdiff/92785dac2ee7026948962cd61c4cd84a2d052772
  • Improve efficiency of dblink by using libpq's new row processor API. This patch provides a test case for libpq's row processor API. contrib/dblink can deal with very large result sets by dumping them into a tuplestore (which can spill to disk) --- but until now, the intermediate storage of the query result in a PGresult meant memory bloat for any large result. Now we use a row processor to convert the data to tuple form and dump it directly into the tuplestore. A limitation is that this only works for plain dblink() queries, not dblink_send_query() followed by dblink_get_result(). In the latter case we don't know the desired tuple rowtype soon enough. While hack solutions to that are possible, a different user-level API would probably be a better answer. Kyotaro Horiguchi, reviewed by Marko Kreen and Tom Lane http://git.postgresql.org/pg/commitdiff/6f922ef88e43b3084cdddf4b5ffe525a00896a90
  • Fix plpgsql named-cursor-parameter feature for variable name conflicts. The parser got confused if a cursor parameter had the same name as a plpgsql variable. Reported and diagnosed by Yeb Havinga, though this isn't exactly his proposed fix. Also, some mostly-but-not-entirely-cosmetic adjustments to the original named-cursor-parameter patch, for code readability and better error diagnostics. http://git.postgresql.org/pg/commitdiff/05dbd4a7734e09bd1f835f4197d9befa1c00c4f3
  • Update obsolete comment. Somebody didn't bother to fix this comment while adding foreign table support to the code below it. In passing, remove the explicit calling-out of relkind letters, which adds complexity to the comment but doesn't help in understanding the code. http://git.postgresql.org/pg/commitdiff/a75b08066ab4b426b1bd056fed81302e5a5d5371
  • Allow statistics to be collected for foreign tables. ANALYZE now accepts foreign tables and allows the table's FDW to control how the sample rows are collected. (But only manual ANALYZEs will touch foreign tables, for the moment, since among other things it's not very clear how to handle remote permissions checks in an auto-analyze.) contrib/file_fdw is extended to support this. Etsuro Fujita, reviewed by Shigeru Hanada, some further tweaking by me. http://git.postgresql.org/pg/commitdiff/263d9de66b867b7800fac82c222e004b795b724a
  • Dept of second thoughts: improve the API for AnalyzeForeignTable. If we make the initially-called function return the table physical-size estimate, acquire_inherited_sample_rows will be able to use that to allocate numbers of samples among child tables, when the day comes that we want to support foreign tables in inheritance trees. http://git.postgresql.org/pg/commitdiff/cea49fe82fedcf125eb99a780099eaf47a326b03
  • Fix broken comparetup_datum code. Commit 337b6f5ecf05b21b5e997986884d097d60e4e3d0 contained the entirely fanciful assumption that it had made comparetup_datum unreachable. Reported and patched by Takashi Yamamoto. Fix up some not terribly accurate/useful comments from that commit, too. http://git.postgresql.org/pg/commitdiff/17b985b1a068e120feb05bb61be6328f4fef8090
  • Fix misleading output from gin_desc(). XLOG_GIN_UPDATE_META_PAGE and XLOG_GIN_DELETE_LISTPAGE records were printed with a list link field labeled as "blkno", which was confusing, especially when the link was empty (InvalidBlockNumber). Print the metapage block number instead, since that's what's actually being updated. We could include the link values too as a separate field, but not clear it's worth the trouble. Back-patch to 8.4 where the dubious code was added. http://git.postgresql.org/pg/commitdiff/0ab4db52c0d45763adee3981da4325e7c353e443
  • Update URL for pgtclng project. Thom Brown http://git.postgresql.org/pg/commitdiff/d75829a682fdf832fe9d044b28be597d3f058ec8
  • Remove useless variable to suppress compiler warning. http://git.postgresql.org/pg/commitdiff/a25ef7a5f68728f7f78f798a98c26b773c45937e
  • Further adjustment of comment about qsort_tuple. http://git.postgresql.org/pg/commitdiff/95b9c333b25463283ba07c5ba7dbe79b4e8b4480
  • Fix incorrect make maintainer-clean rule. http://git.postgresql.org/pg/commitdiff/7feecedccef078bb9f39188db2f75c1481b45cc3
  • Improve management of "sticky" entries in contrib/pg_stat_statements. This patch addresses a deficiency in the previous pg_stat_statements patch. We want to give sticky entries an initial "usage" factor high enough that they probably will stick around until their query is completed. However, if the query never completes (eg it gets an error during execution), the entry shouldn't persist indefinitely. Manage this by starting out with a usage setting equal to the (approximate) median usage value within the whole hashtable, but decaying the value much more aggressively than we do for normal entries. Peter Geoghegan http://git.postgresql.org/pg/commitdiff/d5375491f8e391224b48e4bb449995a4642183ea

Simon Riggs a poussé :

Bruce Momjian a poussé :

Heikki Linnakangas a poussé :

  • Do stack-depth checking in all postmaster children. We used to only initialize the stack base pointer when starting up a regular backend, not in other processes. In particular, autovacuum workers can run arbitrary user code, and without stack-depth checking, infinite recursion in e.g an index expression will bring down the whole cluster. The comment about PL/Java using set_stack_base() is not yet true. As the code stands, PL/java still modifies the stack_base_ptr variable directly. However, it's been discussed in the PL/Java mailing list that it should be changed to use the function, because PL/Java is currently oblivious to the register stack used on Itanium. There's another issues with PL/Java, namely that the stack base pointer it sets is not really the base of the stack, it could be something close to the bottom of the stack. That's a separate issue that might need some further changes to this code, but that's a different story. Backpatch to all supported releases. http://git.postgresql.org/pg/commitdiff/ef3883d130cb553003015f3da7fa59415da6f5c9
  • set_stack_base() no longer needs to be called in PostgresMain. This was a thinko in previous commit. Now that stack base pointer is now set in PostmasterMain and SubPostmasterMain, it doesn't need to be set in PostgresMain anymore. http://git.postgresql.org/pg/commitdiff/03529a3ff999b0e4770b3e0b8e342400e383db98

Correctifs rejetés (à ce jour)

  • Pas de déception cette semaine :-)

Correctifs en attente

  • Etsuro Fujita and Shigeru HANADA traded patches to implement various parts of a PostgreSQL FDW.
  • Gilles Darold sent in some patches to help tell when PostgreSQL is in backup mode.
  • Josh Kupershmidt sent in a patch to fix psql's tab completion for WITH under some circumstances.
  • Heikki Linnakangas and Pavel Stehule traded versions of the CHECK TRIGGER patch.
  • Joachim Wieland sent in two more revisions of the patch to make pg_dump able to operate in parallel.
  • Yeb Havinga sent in two revisions of a patch intended to fix an issue with cursor arguments in named notation.
  • Zoltan Boszormenyi sent in five revisions of a patch to implement a framework to simplify and correctly nest multiplexing more than two timeout sources into the same SIGALRM signal handler.
  • Robert Haas sent in a patch to fix the SLRU replacement algorithm.
  • Peter Eisentraut sent in a patch to rename bytea_agg() to string_agg(), which now covers bytea types.
  • Peter Geoghegan sent in a patch to do some speedups in pg_stat_statements.
  • Alvaro Herrera sent in another revision of the patch to add URI support to libpq, along with some documentation fo the changes.

par N Bougain le jeudi 19 avril 2012 à 23h26

mardi 17 avril 2012

Jean-Paul Argudo

PGDay France 2012 : le 7 juin à Lyon

Bonjour à tous,

Le PG Day France 2012 aura lieu cette année à Lyon le 7 juin 2012, dans les locaux de l'INSA qui nous accueillent cette année.

Pour les conférences, le programme est en ligne. Nous sommes en train de décider des ordres de passage.

Comme d'accoutumée, l’événement est gratuit pour les étudiants de l'école qui nous accueille. Aussi, étudiants de l'INSA à Lyon, n'hésitez pas à vous inscrire et peut-être découvrir PostgreSQL. Il en va bien sûr de même pour les professeurs et chercheurs de l'INSA.

Chers étudiants, à n'en pas douter, au cours de votre carrière, vous serez confrontés tôt ou tard à ce SGBD, étant donné son adoption toujours plus grandissante d'année en année, et ce, depuis le début des années 2000.

Tout le monde doit s'inscrire sur la page des inscriptions, quel que soit son profil. Il vous en coûtera au maximum 30€ pour la journée, repas inclus.

Étant donné la qualité des intervenants cette année encore, je n'ai qu'un conseil à vous donner: inscrivez-vous vite avant qu'il n'y ait plus de places !

... à bon entendeur, salut !

par Jean-Paul le mardi 17 avril 2012 à 12h14

mercredi 11 avril 2012

Dimitri Fontaine (2nd Quadrant)

Migration de MySQL vers PostgreSQL

La conférence annuelle des développeurs PostgreSQL a lieu cette année encore à Ottawa, au Canada, et nous y serons présent. J’aurai le plaisir de raconter une migration MySQL effectuée cet hivers, pour un site de type réseau social dont l’heure de gloire l’a placé dans les 10 sites les plus visités sur internet.

Il s’agit de Fotolog, qui anime de très grandes communautés d’utilisateurs en amérique du sud et partout ailleurs dans le monde. Avec 32 millions d’utilisateurs, 1 milliard de photo et 10 milliards de commentaires, la migration à PostgreSQL était intéressante et mérite de s’y arrêter le temps d’un article et d’une conférence.

Nous parlons souvent de la versatilité de PostgreSQL : cette migration a montré à quel point il est important de pouvoir compter sur cette qualité du projet. En effet, nous avons pu utiliser l’intégration de Java en tant que language de procédure stockée (PL/Java) afin de traiter certains BLOBs et de les normaliser en les intégrant à nouveau dans le schéma de la base de données, par exemple.

Il est par ailleurs intéressant de noter que si les raisons pour lesquelles nous avons changé de MySQL pour lui préférer PostgreSQL sur ce site à fort trafic ne sont pas essentiellement techniques, les deux principaux acteurs des réseaux sociaux Facebook et Twitter ont chacun publié une série de correctifs (patches).

Ils utilisent donc en production une version personnalisée de MySQL, et dont seules les sources sont distribuées, sans soucis de maintenance tierce aucuns. Est-il nécessaire de constituer une équipe de développeurs de moteur de base de données et de maintenir sa propre version en interne afin de pouvoir utiliser MySQL pour de fortes volumétries ?

par Dimitri Fontaine le mercredi 11 avril 2012 à 13h51

mardi 10 avril 2012

Actualités PostgreSQL.fr

Sortie de PostGIS 2.0

Le groupe de développement de PostGIS vient d'annoncer la sortie officielle de la version 2.0 de PostGIS.

Vous pouvez dès maintenant télécharger et installer cette version en vous rendant sur la page de téléchargement du projet :

http://postgis.org/download/

Pour rappel, PostGIS est la cartouche spatiale de PostgreSQL, la base de donnée open source relationnelle la plus avancée. Le couple PostgreSQL/PostGIS est souvent la pierre angulaire des systèmes d'information géographique. PostGIS apporte de nouveaux types de données (points, lignes, polygones…), un mécanisme d'indexation spatial, et un grand nombre de fonctions pour travailler avec ces données.

PostGIS 2.0, qui vient donc de voir le jour, arrive après un peu plus de 2 ans de développement. Les améliorations sont nombreuses, tant en terme de fonctionnalités, que de changements dans le code interne de PostGIS. Cette version utilise également les bibliothèques GEOS 3.3.3 et GDAL 1.9.0, qui sont sorties récemment.

Parmi les grandes nouvelles fonctionnalités, on trouve la gestion des Raster (données image) dans la base de données, ainsi que la gestion d’un modèle topologique respectant le standard SQL/MM. Mais ce n’est pas tout, et voici la liste des fonctionnalités introduites par PostGIS 2.0 :

  • Gestion des données raster et analyse raster/vecteur en base de données
  • Modèle topologique pour gérer les objets avec des frontières communes (pavages de plan par exemple)
  • Intégration du typmod PostgreSQL, avec table geometry_columns automatique
  • Indexation 3D et 4D
  • Recherche de plus proches voisins optimisée grâce à l’utilisation de l'indexation (KNN-search)
  • De nombreuses nouvelles fonctions de traitement vectoriel, dont : ST_Split,ST_Node, ST_MakeValid, ST_OffsetCurve, ST_ConcaveHull, ST_AsX3D, ST_GeomFromGeoJSON, ST_3DDistance
  • Utilisation du mécanisme d’extension de PostgreSQL 9.1
  • Améliorations sur l’outil de chargement/sauvegarde de shapefiles en ligne de commande
  • Gestion multi fichier pour l’import et l’export dans l’outil d’interface graphique
  • Un géocodeur pour les données US Census TIGER 2010
  • Gestion initiale de primitives 3D

En outre, de nombreuses améliorations et refactorisations ont été faites dans le cœur de PostGIS, rendant cet outil un des plus performants du marché.

Tous les membres de l'équipe de développement de PostGIS tiennent à remercier leurs parents d'avoir rendu cette sortie possible.

Quelques liens :

(Merci à Vincent Picavet pour la traduction)

par daamien le mardi 10 avril 2012 à 09h09

samedi 31 mars 2012

Yoran Brault

Convertir un site Drupal MySQL vers PostgreSQL simplement

Convertir un site Drupal MySQL vers PostgreSQL simplement

200px-postgresql_elephant.svg_.png

Combien de fois ais-je entendu "Postgres c'est super lent comparé à MySQL" (version polie). Et ce qui est amusant c'est que généralement cette affirmation s'appuie uniquement sur un ressenti de développeur assis devant sa bécane de compétition qui n'aura donc jamais qu'un seul pauvre utilisateur à servir... Assez faible vous en conviendrez. Le problème c'est que pour faire une vraie comparaison, il faudrait travailler sur un vrai site qui existerait pour les deux bases de données. Un cas de test généralement délicat voir impossible à monter tant les SGBDR sont mauvais dés qu'il s'agit de les interchanger.

Partant de cela, je me suis demandé, dans le cas exclusif d'un site sous Drupal, comment permettre une telle conversion. Et j'ai fini par tomber sur une méthode finalement assez simple.

Déjà, pourquoi est-ce normalement compliqué ?

En effet, puisque tout SGBDR utilise le langage SQL, convertir une application devrait être l'enfance de l'art. Malheureusement, ça c'est sur le papier. Dans la réalité, c'est un peu comme la norme CSS et Internet Explorer, chaque éditeur de base de données y vas de ses exotismes tant et si bien qu'à la fin, l'application se retrouve pieds et poings liés. Si l'on réduit le champ à une application PHP, les problèmes sont les suivants :

  • Les fonctions PHP (connexion, requêtes, etc) utilisées sont différentes d'une base à l'autre.
  • Les créations de tables ne fonctionnent que pour un moteur de base de données.
  • Les requêtes utilisent des fonctions spécifiques et même parfois des syntaxes propres à une base.

En bref, tester les performances sous PostgreSQL d'une application PHP écrite pour MySQL, c'est même pas la peine d'y penser?

Abstraction

C'est pour cela que Drupal, comme une majorité de cadres applicatifs, cherche depuis bien longtemps à mettre en place une certaine forme d'abstraction de la couche de stockage, adressant les trois aspects vu plus haut : la connexion et l'échange de données, la création de tables et le requêtage.

Drupal étant assez simpliste (euphémisme) dans son usage d'une base (pas de clefs étrangères, pas de procédures stockées, pas de triggers, pas de vues, etc), il s'est longtemps borné avec succès à l'abstraction de la connexion et ce principalement pour deux bases : PostgreSQL et MySQL. Avec Drupal 7 et l'adoption de PDO, cette aspect a été encore grandement amélioré.

Depuis Drupal6, Schema API adresse l'épineux problème des disparités dans la syntaxe de création des tables. Il est depuis cette version possible d'écrire des modules qui créent des tables dans une sorte de langage pivot (dans le plus pur style d'Array Oriented Programming Drupalien) laissant au système le soin de le traduire dans celui de la base utilisée.

Enfin, avec Drupal 7, arrive l'excellente Database API qui vient compléter l'attirail en proposant l'abstraction des requêtes dynamiques.

En somme, depuis Drupal 7, nous pouvons donc, avec un peu de soin et de méthode, commencer à écrire des modules qui sont ainsi quasiment agnostiques concernant la couche de stockage.

Schema API et conversion de base

Pour revenir à notre question d'origine, dans la mesure où Drupal peut facilement basculer d'une base à l'autre (abstraction de la connexion), convertir une base de donnée Drupal de MySQL à PostgreSQL se simplifie donc à la résolution de deux problématiques :

  • Créer une base PostgreSQL content l'ensemble des tables de notre installation MySQL.
  • Injecter dans PostgreSQL un dump provenant de MySQL.

Si le second point ne semble pas bien sorcier (ce ne sont au fond juste que des insert), la création du schéma de la base est autrement plus épineux car il s'agit littéralement de traduire le langage MySQL en langage PostgreSQL. Bon courage...

C'est là que Schema API vient à notre rescousse. En effet, à quoi sert de convertir le schéma de MySQL puisque drupal depuis sa version 6 dispose pour l'ensemble de ses modules activés d'une description précise des tables créées. La seule astuce qu'il nous faut trouver, c'est comment le forcer à produire du code "PostgreSQL" lorsqu'il travaille avec MySQL.

Problématique résolue par l'usage de l'indispensable module Schema. Ce module a été écrit pour lister les schémas définis par chaque module installés. Il est surtout utilisé pour vérifier que la base de donnée physique est bien en accord avec la description que drupal en a gardé et permet notamment de faire du ménage sur des tables laissées sans objets par exemple lors de mises à jour.

Mais schema permet aussi de générer un script SQL de création de l'ensemble des tables répertoriées. Premier pas vers notre conversion qui s'arrête rapidement car il se limite à utiliser la syntaxe du moteur en cours d'usage.

Pour contourner cela, nous allons utiliser les fonctionnalités du module Schema dans un petit script très simple :

#! /usr/bin/php
<?php
define('DRUPAL_ROOT', getcwd());

require_once DRUPAL_ROOT . '/includes/bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);

$schema = drupal_get_schema(NULL, TRUE);
$sql = '';
$class_name = 'SchemaDatabaseSchema_pgsql';
$engine = new $class_name(Database::getConnection());
foreach ($schema as $name => $table) {
  if (substr($name, 0, 1) == '#') {
    continue;
  }
  $stmts = $engine->getCreateTableSql($name, $table);
  $sql .= implode(";\n", $stmts) . ";\n\n";
}
echo str_replace(array('{', '}'), '', $sql);
dump_schema.php

Tout le début n'est qu'une recopie du script PHP de démarrage de Drupal (index.php). Ce qui suit le boostrap est la fonction du module Schema modifiée de sorte à forcer l'usage du traducteur PostgreSQL et ainsi produire un script SQL pour ce moteur.

Il suffit ensuite de lancer ce script pour avoir régler 80% de notre problématique avec code SQL généré en sortie standard qu'il suffira d'injecter dans PostgreSQL. Ensuite il ne restera plus qu'à injecter les données elles-mêmes. La procédure complète donne dés lors quelque chose comme ceci :

# Création de la base de donnée sur postgres
gaston$echo "create database mon_site" | psql -Upostgres
 
# Injection du schema
gaston$php ./dump_schema.php | psql -Upostgres mon_site
 
# Injection des données
gaston$mysqldump -uroot -pmot_de_passe_root --no-create-info --compatible=postgresql mon_site | sed "s/\\\'/\'\'/g" | psql -Upostgres mon_site

Avouez que ce n'est pas sorcier, et le pire c'est que cela fonctionne. La dernière ligne de la procédure, un peu longuette, permet de générer un dump MySQL contenant seulement les inserts (--no-create-info) et formalisé pour PostgreSQL (--compatible=postgresql). La commande SED est juste là pour convertir les \' en ''. L'exécution de cet import va générer un paquet de warning car PostgreSQL aimerait bien que l'on utilise un approche standard pour les autres échappements (\r, \n, etc.) mais au final, il ne faut que crier et les données sont bien insérées.

Conclusion

Cette méthode est grandement améliorable, automatisable et généralisable à d'autres échanges. Je verrais si j'ai le temps d'écrire une procédure plus générique. Mais déjà en l'état j'ai pu la tester sur un gros site en Drupal 7/MySQL qui s'est mis à tourner sous PostgreSQL sans autre tracas. Magique.

J'ai à cette occasion pu constater que les performances de MySQL teeeeellllement meilleurs que PostgreSQL était pipo/clarinette mais ça, c'est une autre histoire :-)

par Yoran le samedi 31 mars 2012 à 15h10

vendredi 30 mars 2012

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 25 mars 2012

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en mars

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA.

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

(lien vers l'article original)

Correctifs appliqués

Bruce Momjian a poussé :

Alvaro Herrera a poussé :

Tom Lane a poussé :

  • Restructure SELECT INTO's parsetree representation into CreateTableAsStmt. Making this operation look like a utility statement seems generally a good idea, and particularly so in light of the desire to provide command triggers for utility statements. The original choice of representing it as SELECT with an IntoClause appendage had metastasized into rather a lot of places, unfortunately, so that this patch is a great deal more complicated than one might at first expect. In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE Alexander Shulgin subcommands required restructuring some EXPLAIN-related APIs. Add-on code that calls ExplainOnePlan or ExplainOneUtility, or uses ExplainOneQuery_hook, will need adjustment. Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO, which formerly were accepted though undocumented, are no longer accepted. The PREPARE case can be replaced with use of CREATE TABLE Alexander Shulgin EXECUTE. The CREATE RULE case doesn't seem to have much real-world use (since the rule would work only once before failing with "table already exists"), so we'll not bother with that one. Both SELECT INTO and CREATE TABLE Alexander Shulgin still return a command tag of "SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn", but for the moment backwards compatibility wins the day. Andres Freund and Tom Lane http://git.postgresql.org/pg/commitdiff/9dbf2b7d75de5af38d087cbe2b1147dd0fd10f0a
  • Improve the -l (limit) option recently added to contrib/vacuumlo. Instead of just stopping after removing an arbitrary subset of orphaned large objects, commit and start a new transaction after each -l objects. This is just as effective as the original patch at limiting the number of locks used, and it doesn't require doing the OID collection process repeatedly to get everything. Since the option no longer changes the fundamental behavior of vacuumlo, and it avoids a known server-side limitation, enable it by default (with a default limit of 1000 LOs per transaction). In passing, be more careful about properly quoting the names of tables and fields, and do some other cosmetic cleanup. http://git.postgresql.org/pg/commitdiff/64c604898e812aa93c124c666e8709fff1b8dd26
  • Back-patch contrib/vacuumlo's new -l (limit) option into 9.0 and 9.1. Since 9.0, removing lots of large objects in a single transaction risks exceeding max_locks_per_transaction, because we merged large object removal into the generic object-drop mechanism, which takes out an exclusive lock on each object to be dropped. This creates a hazard for contrib/vacuumlo, which has historically tried to drop all unreferenced large objects in one transaction. There doesn't seem to be any correctness requirement to do it that way, though; we only need to drop enough large objects per transaction to amortize the commit costs. To prevent a regression from pre-9.0 releases wherein vacuumlo worked just fine, back-patch commits b69f2e36402aaa222ed03c1769b3de6d5be5f302 and 64c604898e812aa93c124c666e8709fff1b8dd26, which break vacuumlo's deletions into multiple transactions with a user-controllable upper limit on the number of objects dropped per transaction. Tim Lewis, Robert Haas, Tom Lane http://git.postgresql.org/pg/commitdiff/5bd06e619c82c3b2e29fed40aae5fc39a9620908
  • Allow new relmapper entries when allow_system_table_mods is true. This restores the pre-9.0 situation that it's possible to add new indexes on pg_class and other mapped-but-not-shared catalogs, so long as you broke the glass and flipped the big red Dont-Touch-Me switch. As before, there are a lot of gotchas, and you'd have to be pretty desperate to try this on a production database; but there doesn't seem to be a reason for relmapper.c to be preventing such things all by itself. Per experimentation with a case suggested by Cody Cutrer. http://git.postgresql.org/pg/commitdiff/f70f095c9096d5e2689e8d79172b37b57a84e51b
  • Fix configure's search for collateindex.pl. PGAC_PATH_COLLATEINDEX supposed that it could use AC_PATH_PROGS to search for collateindex.pl, but that macro will only accept files that are marked executable, and at least some DocBook installations don't mark the script executable (a case the docs Makefile was already prepared for). Accept the script if it's present and readable in $DOCBOOKSTYLE/bin, and otherwise search the PATH as before. Having fixed that up, we don't need the fallback case that was in the docs Makefile, and instead can throw an understandable error if configure didn't find the script. Per recent trouble report from John Lumby. http://git.postgresql.org/pg/commitdiff/f2386d7136dacbb8bf59ebbe8e5c5b73db202813
  • If a role has a password expiration date, show that in psql's \du output. Per a suggestion from Euler Taveira, it seems like a good idea to include this information in \du (and \dg) output. This costs nothing for people who are not using the VALID UNTIL feature, while for those who are, it's rather critical information. Fabrízio de Royes Mello http://git.postgresql.org/pg/commitdiff/04dfc8774966c60e4031ab5d8eeb3263d9568ad3
  • Refactor to eliminate duplicate copies of conninfo default-finding code. Alexander Shulgin, lightly edited by me. http://git.postgresql.org/pg/commitdiff/e9ce658b6203e9052bff5c89ed6265fba69cdb36
  • Fix GET DIAGNOSTICS for case of assignment to function's first variable. An incorrect and entirely unnecessary "safety check" in exec_stmt_getdiag() caused the code to treat an assignment to a variable with dno zero as a no-op. Unfortunately, that's a perfectly valid dno. This has been broken since GET DIAGNOSTICS was invented. It's not terribly surprising that the bug went unnoticed for so long, since in most cases you probably wouldn't use the function's first-created variable (normally its first parameter) as a GET DIAGNOSTICS target. Nonetheless, it's broken. Per bug #6551 from Adam Buraczewski. http://git.postgresql.org/pg/commitdiff/88a4cb30a4c48a6b11fd4641855595d2678aa123
  • Code review for protransform patches. Fix loss of previous expression-simplification work when a transform function fires: we must not simply revert to untransformed input tree. Instead build a dummy FuncExpr node to pass to the transform function. This has the additional advantage of providing a simpler, more uniform API for transform functions. Move documentation to a somewhat less buried spot, relocate some poorly-placed code, be more wary of null constants and invalid typmod values, add an opr_sanity check on protransform function signatures, and some other minor cosmetic adjustments. Note: although this patch touches pg_proc.h, no need for catversion bump, because the changes are cosmetic and don't actually change the intended catalog contents. http://git.postgresql.org/pg/commitdiff/0339047bc93147c1c6f78f867ae6b0c215406235
  • Refactor simplify_function et al to centralize argument simplification. We were doing the recursive simplification of function/operator arguments in half a dozen different places, with rather baroque logic to ensure it didn't get done multiple times on some arguments. This patch improves that by postponing argument simplification until after we've dealt with named parameters and added any needed default expressions. Marti Raudsepp, somewhat hacked on by me http://git.postgresql.org/pg/commitdiff/81a646febe87964725647a36d839f6b4b405f3ae
  • Cast some printf arguments to avoid possibly-nonportable behavior. Per compiler warnings on buildfarm member black_firefly. http://git.postgresql.org/pg/commitdiff/ed61127be483d8939e13a151773549f1517d6e67
  • Fix planner's handling of outer PlaceHolderVars within subqueries. For some reason, in the original coding of the PlaceHolderVar mechanism I had supposed that PlaceHolderVars couldn't propagate into subqueries. That is of course entirely possible. When it happens, we need to treat an outer-level PlaceHolderVar much like an outer Var or Aggref, that is SS_replace_correlation_vars() needs to replace the PlaceHolderVar with a Param, and then when building the finished SubPlan we have to provide the PlaceHolderVar expression as an actual parameter for the SubPlan. The handling of the contained expression is a bit delicate but it can be treated exactly like an Aggref's expression. In addition to the missing logic in subselect.c, prepjointree.c was failing to search subqueries for PlaceHolderVars that need their relids adjusted during subquery pullup. It looks like everyplace else that touches PlaceHolderVars got it right, though. Per report from Mark Murawski. In 9.1 and HEAD, queries affected by this oversight would fail with "ERROR: Upper-level PlaceHolderVar found where not expected". But in 9.0 and 8.4, you'd silently get possibly-wrong answers, since the value transmitted into the subquery wouldn't go to null when it should. http://git.postgresql.org/pg/commitdiff/8279eb4191c7ab9920c72ec8eec5df0e7b8c7530
  • Replace empty locale name with implied value in CREATE DATABASE and initdb. setlocale() accepts locale name "" as meaning "the locale specified by the process's environment variables". Historically we've accepted that for Postgres' locale settings, too. However, it's fairly unsafe to store an empty string in a new database's pg_database.datcollate or datctype fields, because then the interpretation could vary across postmaster restarts, possibly resulting in index corruption and other unpleasantness. Instead, we should expand "" to whatever it means at the moment of calling CREATE DATABASE, which we can do by saving the value returned by setlocale(). For consistency, make initdb set up the initial lc_xxx parameter values the same way. initdb was already doing the right thing for empty locale names, but it did not replace non-empty names with setlocale results. On a platform where setlocale chooses to canonicalize the spellings of locale names, this would result in annoying inconsistency. (It seems that popular implementations of setlocale don't do such canonicalization, which is a pity, but the POSIX spec certainly allows it to be done.) The same risk of inconsistency leads me to not venture back-patching this, although it could certainly be seen as a longstanding bug. Per report from Jeff Davis, though this is not his proposed patch. http://git.postgresql.org/pg/commitdiff/c7cea267de3ca05b29a57b9d113b95ef3793c8d8
  • Fix COPY FROM for null marker strings that correspond to invalid encoding. The COPY documentation says "COPY FROM matches the input against the null string before removing backslashes". It is therefore reasonable to presume that null markers like E'\\0' will work ... and they did, until someone put the tests in the wrong order during microoptimization-driven rewrites. Since then, we've been failing if the null marker is something that would de-escape to an invalidly-encoded string. Since null markers generally need to be something that can't appear in the data, this represents a nontrivial loss of functionality; surprising nobody noticed it earlier. Per report from Jeff Davis. Backpatch to 8.4 where this got broken. http://git.postgresql.org/pg/commitdiff/e8476f46fc847060250c92ec9b310559293087fc

Peter Eisentraut a poussé :

Robert Haas a poussé :

Correctifs rejetés (à ce jour)

  • Pas de déception cette semaine :-)

Correctifs en attente

  • Marco Nenciarini sent in another revision of the patch to support arrays each element of which is a foreign key.
  • Andres Freund sent in a patch to check that the specified tablespace in a CREATE TABLE AS command is not pg_global.
  • Daniel Farina sent in another revision of the patch to allow users to terminate their own backends.
  • Dimitri Fontaine sent in another revision of the patch to implement command triggers.
  • Robert Haas sent in a patch to remove a dead code path in heap_freeze_tuple().
  • Dimitri Fontaine sent in three more revisions of the patch to make finer EXTENSION dependencies.
  • Ants Aasma sent in another revision of the patch to add timing of buffer I/O requests.
  • Peter Geoghegan sent in another revision of the patch to publish checkpoint timing and sync files summary data to pg_stat_bgwriter.
  • Alexander Shulgin sent in another revision of the patch to add a URI connection string to libpq.
  • Peter Eisentraut sent in another revision of the patch to allow renaming a DOMAIN constraint.
  • Marko Kreen sent in another revision of the patch to create a new, more efficient tuple storage in libpq and use same in dblink.
  • David Fetter sent in two more revisions of a patch to allow CREATE FOREIGN TABLE to take a LIKE clause.
  • Fujii Masao sent in three revisions of a patch to change pg_controldata so that it reports the name of WAL file containing the latest checkpoint's REDO record.
  • Zoltan Boszormenyi sent in another revision of the patch to enable ECPG fetch readahead.
  • Joachim Wieland sent in another revision of the patch to enable parallel pg_dump.
  • Jeff Davis sent in another revision of the patch to fix initdb's behavior with fsync.
  • Peter Geoghegan sent in another revision of the patch to allow pg_stat_statements to be normalized.

par N Bougain le vendredi 30 mars 2012 à 07h21

Nouvelles hebdomadaires de PostgreSQL - 18 mars 2012

La première réunion du PUG d'Arizona est programmée pour le 29 mars 2012 à 17h. Au menu : la présentation de Ken Rosensteel, mise à jour depuis le PGEast 2011, avec l'ajout des retours d'expérience de Bull sur l'utilisation de PostgreSQL par de gros clients. Vous pouvez rejoindre la mailing-list, consulter l'agenda et RSVP sur : https://www.bigtent.com/groups/azpug

Réunion du PUG londonien le 17 avril 2012 : http://www.meetup.com/London-PostgreSQL-Meetup-Group/

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en mars

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA.

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

(lien vers l'article original)

Revues de code

Correctifs appliqués

Bruce Momjian a poussé :

Peter Eisentraut a poussé :

Tom Lane a poussé :

  • Fix SPGiST vacuum algorithm to handle concurrent tuple motion properly. A leaf tuple that we need to delete could get moved as a consequence of an insertion happening concurrently with the VACUUM scan. If it moves from a page past the current scan point to a page before, we'll miss it, which is not acceptable. Hence, when we see a leaf-page REDIRECT that could have been made since our scan started, chase down the redirection pointer much as if we were doing a normal index search, and be sure to vacuum every page it leads to. This fixes the issue because, if the tuple was on page N at the instant we start our scan, we will surely find it as a consequence of chasing the redirect from page N, no matter how much it moves around in between. Problem noted by Takashi Yamamoto. http://git.postgresql.org/pg/commitdiff/b4af1c25bbc636379efc5d2ffb9d420765705b8a
  • Create a stack of pl/python "execution contexts". This replaces the former global variable PLy_curr_procedure, and provides a place to stash per-call-level information. In particular we create a per-call-level scratch memory context. For the moment, the scratch context is just used to avoid leaking memory from datatype output function calls in PLyDict_FromTuple. There probably will be more use-cases in future. Although this is a fix for a pre-existing memory leakage bug, it seems sufficiently invasive to not want to back-patch; it feels better as part of the major rearrangement of plpython code that we've already done as part of 9.2. Jan Urbański http://git.postgresql.org/pg/commitdiff/ed75380bdae30dc1313aef44beafad860cf246c0
  • Fix minor memory leak in PLy_typeinfo_dealloc(). We forgot to free the per-attribute array element descriptors. Jan Urbański http://git.postgresql.org/pg/commitdiff/a14fa84693659c4c4a17204406945b29fae3d9c4
  • Patch some corner-case bugs in pl/python. Dave Malcolm of Red Hat is working on a static code analysis tool for Python-related C code. It reported a number of problems in plpython, most of which were failures to check for NULL results from object-creation functions, so would only be an issue in very-low-memory situations. Patch in HEAD and 9.1. We could go further back but it's not clear that these issues are important enough to justify the work. Jan Urbański http://git.postgresql.org/pg/commitdiff/5cd72c7a7c7bd76ab028e1dc59d90a47750acebe
  • Revisit handling of UNION ALL subqueries with non-Var output columns. In commit 57664ed25e5dea117158a2e663c29e60b3546e1c I tried to fix a bug reported by Teodor Sigaev by making non-simple-Var output columns distinct (by wrapping their expressions with dummy PlaceHolderVar nodes). This did not work too well. Commit b28ffd0fcc583c1811e5295279e7d4366c3cae6c fixed some ensuing problems with matching to child indexes, but per a recent report from Claus Stadler, constraint exclusion of UNION ALL subqueries was still broken, because constant-simplification didn't handle the injected PlaceHolderVars well either. On reflection, the original patch was quite misguided: there is no reason to expect that EquivalenceClass child members will be distinct. So instead of trying to make them so, we should ensure that we can cope with the situation when they're not. Accordingly, this patch reverts the code changes in the above-mentioned commits (though the regression test cases they added stay). Instead, I've added assorted defenses to make sure that duplicate Emmanuel Cecchet child members don't cause any problems. Teodor's original problem ("MergeAppend child's targetlist doesn't match MergeAppend") is addressed more directly by revising prepare_sort_from_pathkeys to let the parent MergeAppend's sort list guide creation of each child's sort list. In passing, get rid of add_sort_column; as far as I can tell, testing for duplicate sort keys at this stage is dead code. Certainly it doesn't trigger often enough to be worth expending cycles on in ordinary queries. And keeping the test would've greatly complicated the new logic in prepare_sort_from_pathkeys, because comparing pathkey list entries against a previous output array requires that we not skip any entries in the list. Back-patch to 9.1, like the previous patches. The only known issue in this area that wasn't caused by the ill-advised previous patches was the MergeAppend planning failure, which of course is not relevant before 9.1. It's possible that we need some of the new defenses against duplicate child Emmanuel Cecchet entries in older branches, but until there's some clear evidence of that I'm going to refrain from back-patching further. http://git.postgresql.org/pg/commitdiff/dd4134ea56cb8855aad3988febc45eca28851cd8
  • Improve commentary in match_pathkeys_to_index(). For a little while there I thought match_pathkeys_to_index() was broken because it wasn't trying to match index columns to pathkeys in order. Actually that's correct, because GiST can support ordering operators on any random collection of index columns, but it sure needs a comment. http://git.postgresql.org/pg/commitdiff/b67ad046e6c37cbe0eebc5745323ed9864192f52

Robert Haas a poussé :

Heikki Linnakangas a poussé :

Andrew Dunstan a poussé :

  • Honor inputdir and outputdir when converting regression files. When converting source files, pg_regress' inputdir and outputdir options were ignored when computing the locations of the destination files. In consequence, these options were effectively unusable when the regression inputs need to be adjusted by pg_regress. This patch makes pg_regress put the converted files in the same place that these options specify non-converted input or results files are to be found. Backpatched to all live branches. http://git.postgresql.org/pg/commitdiff/e3fc4a97bc8ee82a78605b5ffe79bd4cf3c6213b

Correctifs rejetés (à ce jour)

  • Pas de déception cette semaine :-)

Correctifs en attente

  • Joachim Wieland sent in two more revisions of the patch to implement parallel pg_dump.
  • Jeff Davis sent in three revisions of a patch to add an initdb option to refrain from doing an fsync.
  • Fujii Masao sent in another revision of the patch add xlog location to pg_size_pretty().
  • Alexander Korotkov sent in a patch to fix an infelicity in GiST indexing on points.
  • Pavel Stehule sent in another revision of the PL/pgsql CHECK FUNCTION code.
  • Fujii Masao sent in a patch to fix a condition which could cause walsender to spin the CPU too much.
  • Etsuro Fujita sent in a patch to support error handling in file_fdw.
  • Hans-Juergen Schoenig sent in another WIP patch to track cross-column correlations.
  • Heikki Linnakangas sent in another revision of the patch to scale xlog insertion.
  • Marco Nenciarini and Gabriele Bartolini traded patches to support arrays each element of which is a foreign key.
  • David Fetter sent in two revisions of a patch to make CREATE FOREIGN TABLE (... LIKE ... ) work.
  • Fabrízio de Royes Mello sent in two revisions of a patch to include VALID UNTIL date in \d for roles (users and groups).
  • Shigeru HANADA sent in another revision of the patch to create a PostgreSQL FDW.
  • Alexander Shulgin sent in another revision of the patch to add a URI format to libpq.
  • Dimitri Fontaine sent in two more revisions of a patch to add command triggers.
  • Daniel Farina sent in three revisions of a patch to make it possible to initiate back-end termination from SQL, race-free.
  • Peter Eisentraut sent in a patch to enable renaming domain constraints.
  • Alvaro Herrera sent in another revision of the patch to implement foreign key locks.
  • Peter Eisentraut sent in a patch to fix incompatible pointer types when pg_dump uses the current (i.e. new) zlib.
  • Joachim Wieland sent in a patch to fix a double free in pg_dump.

par N Bougain le vendredi 30 mars 2012 à 07h13

mercredi 28 mars 2012

Base-sql.fr

Avis sur Postgresql Gestion des performances

Bonjour,

ça fait longtemps qu’il est sorti mais je me devais de faire une critique sur ce livre, qui pour moi est une référence.
Un administrateur en base de données se doit d’avoir au moins lu ce livre, et un administrateur système devrais y jeter un coup d’ oeil au minimum.
Comme beaucoup je suis les deux, souvent les petites entreprises n’ont pas les moyens d’avoir un dba et un admin système, ce livre devient donc une vraie mine d’or (pour pas cher :p).
On y retrouve vraiment tout, on passe du matériel à l’optimisation de requêtes en passant par le monitoring et la maintenance…

Alors biensûr ce livre n’est pas fait pour les débutants, rien que la partie sur le matériel est loin d’être anodine, ça va loin et c’est bien, souvent ce genre de livre n’en parle pas ou survole… ici on a le droit a un bon gros chapitre bien complexe.

Certains chapitres m’ont beaucoup plus, comme l’optimisation des requêtes qui restent complexes mais très passionnants, et qui nous invitent à faire beaucoup de tests et de recherches pour comprendre toutes les subtilités de l’explain.
Le chapitre sur la maintenance est aussi très intéressant tout comme celui parlant des statistiques.

Pour conclure ce livre est une vraie bible, enfin surtout la mienne ! Ce livre vous incite à comprendre ce que vous faites, ce n’est pas qu’un listing de scripts, de requêtes ou de formules, si vous désirez comprendre ce que vous faites ce livre vous y aidera.
Ce livre est un formidable allié pour ceux qui désire se lancer dans l’aventure postgresql et qui désire aller plus loin.

Merci à Gregory Smith et nos 2 traducteurs ( Guillaume Lelarge et Thomas Reiss )

couverture

Lien pour l’acheter chez pearson.fr

par kenrio le mercredi 28 mars 2012 à 14h10

mardi 20 mars 2012

Damien Clochard

Faites circuler l'info : le Google Summer of Code approche !

Si vous avez des connexions dans le milieu universitaire, dans votre ancienne école d'ingénieur ou dans un laboratoire de recherche informatique, je vous encourage à leur transmettre le message ci-dessous.

Cette année encore, la communauté PostgreSQL participe au projet "Summer of Code" (GSoC) de Google qui permet à des étudiants d'obtenir des bourses pour développer des fonctionnalités pour PostgreSQL et ses logiciels satellites.

Pour rappel, voici quelques exemples de développements réalisés dans le cadre des GSoC précédents :

  • Amélioration des Foreign Data Wrappers
  • Nouvelle architecture de plug-in pour phpPgAdmin
  • Concepteur de Schéma pour pgAdmin
  • Extension de pgTune
  • Indexation des données XML dans PostgreSQL
  • Gestion de cache dans pgPool

A noter que ce programme est en langue anglaise mais certains tuteurs ("mentors") sont francophones.

Ci-dessous un exemple de message que vous pouvez transmettre à un professeur ou à un laboratoire de recherche :

http://archives.postgresql.org/pgsq...

par damien le mardi 20 mars 2012 à 09h20

jeudi 15 mars 2012

Actualités PostgreSQL.fr

Créer des index pour les clés étrangères

Aujourd'hui, un client a découvert que les clés étrangères ne créaient pas d'index sur la table sur laquelle on crée la clé étrangère. La table sur la clé primaire de laquelle la clé étrangère pointe est bien sûr indexée.

La question a donc rapidement été: comment créer tous les index manquants, de façon plus ou moins automatisée.

Voici la requête (avec 2 ou 3 explications):

SELECT DISTINCT 'CREATE INDEX CONCURRENTLY ON ' || relname || '(' || array_to_string(dalibo_int_to_colname(pg_class.oid,conkey),',') || ');'
FROM pg_constraint JOIN pg_class ON pg_constraint.conrelid=pg_class.oid 
JOIN pg_namespace ON (pg_class.relnamespace=pg_namespace.oid)
WHERE contype = 'f'
AND nspname not in ('pg_catalog','information_schema') 
AND NOT EXISTS (SELECT indrelid,indkey from pg_index WHERE (indrelid,(indkey::int2[])[0:100])=(pg_constraint.conrelid,pg_constraint.conkey));
  • On ne prend que les foreign keys (contype='f')
  • On ne prend que ce qui est dans les schémas utilisateurs ( nspname not in ('pg_catalog','information_schema') )
  • On ne prend que les contraintes pour lesquelles il n'y a pas déjà un index:
NOT EXISTS (SELECT indrelid,indkey from pg_index WHERE (indrelid,(indkey::int2[])[0:100])=(pg_constraint.conrelid,pg_constraint.conkey));

Ici, on a la première bizarrerie: indkey est de type int2vector, conkey est de type int2.

Les types sont équivalents, à deux points de détails:

  • Les vecteurs commencent à 0,
  • On peut caster de vector vers tableau, mais pas dans l'autre sens

Donc quand on cast l'indkey vers int2, on récupère un tableau qui commence à 0, alors que le tableau de conkey commence à 1. D'où l'extraction de slice 0:100: on demande à PostgreSQL de nous recréer un tableau à partir du tableau indkey::int2, en prenant les éléments de 0 à 100 (il peut y en avoir 32 par défaut, c'est le paramètre de compilation max_index_keys). On récupère donc un tableau indexé à partir de 1, qu'on peut enfin comparer à conkey. indkey est un int2vector pour des raisons de compatibilité (c'est l'ancien type tableau de PostgreSQL).

On a fait le plus pénible: on a la liste des contraintes, sur quelles colonnes de quelles tables elles pointent. Il ne nous reste plus qu'à convertir cette liste de colonne en ordre SQL. Pour cela, il faut passer de l'oid de la table et des identifiants des colonnes à une liste texte des colonnes. Le plus lisible est encore de faire une fonction PL:

CREATE FUNCTION dalibo_int_to_colname(oid_table oid, num_col integer[] )
 RETURNS text[]
 LANGUAGE plpgSQL
 
AS $function$
DECLARE
  velement int;
  varray text[];
  vattname text;
BEGIN
  FOR velement IN SELECT unnest(num_col) LOOP
    SELECT attname INTO vattname FROM pg_attribute WHERE attrelid=oid_table AND attnum=velement;
    varray:=array_append(varray,vattname);
  END LOOP;
  RETURN varray;
END
$function$
;

Pour finir, il y a un DISTINCT parce que plusieurs FOREIGN KEYS peuvent pointer sur le même groupe de colonne.

Une fois que vous avez fini, vérifiez que vous n'avez pas créé des index qui ne servent à rien : http://blog.postgresql.fr/index.php?post/2011/02/22/D%C3%A9tecter-les-index-redondants-dans-une-base

par mcousin le jeudi 15 mars 2012 à 15h00

mardi 13 mars 2012

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 11 mars 2012

Le programme des conférences du PGDay NYC 2012 est disponible : http://pgday.nycpug.org/schedule/

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en mars

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA.

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

(lien vers l'article original)

Revues de code

Correctifs appliqués

Peter Eisentraut a poussé :

Tom Lane a poussé :

  • Improve documentation around logging_collector and use of stderr. In backup.sgml, point out that you need to be using the logging collector if you want to log messages from a failing archive_command script. (This is an oversimplification, in that it will work without the collector as long as you're not sending postmaster stderr to /dev/null; but it seems like a good idea to encourage use of the collector to avoid problems with multiple processes concurrently scribbling on one file.) In config.sgml, do some wordsmithing of logging_collector discussion. Per bug #6518 from Janning Vygen http://git.postgresql.org/pg/commitdiff/3f47e145f1869f147a807e5a2cb80d21a13e10ae
  • Redesign PlanForeignScan API to allow multiple paths for a foreign table. The original API specification only allowed an FDW to create a single access path, which doesn't seem like a terribly good idea in hindsight. Instead, move the responsibility for building the Path node and calling add_path() into the FDW's PlanForeignScan function. Now, it can do that more than once if appropriate. There is no longer any need for the transient FdwPlan struct, so get rid of that. Etsuro Fujita, Shigeru Hanada, Tom Lane http://git.postgresql.org/pg/commitdiff/6b289942bfdbbfa2955cedc591c522822a7ffbfe
  • Add a hook for processing messages due to be sent to the server log. Use-cases for this include custom log filtering rules and custom log message transmission mechanisms (for instance, lossy log message collection, which has been discussed several times recently). As is our common practice for hooks, there's no regression test nor user-facing documentation for this, though the author did exhibit a sample module using the hook. Martin Pihlak, reviewed by Marti Raudsepp http://git.postgresql.org/pg/commitdiff/19dbc3463161a142537ba5c569c8e6a073a318de
  • Expose an API for calculating catcache hash values. Now that cache invalidation callbacks get only a hash value, and not a tuple TID (per commits 632ae6829f7abda34e15082c91d9dfb3fc0f298b and b5282aa893e565b7844f8237462cb843438cdd5e), the only way they can restrict what they invalidate is to know what the hash values mean. setrefs.c was doing this via a hard-wired assumption but that seems pretty grotty, and it'll only get worse as more cases come up. So let's expose a calculation function that takes the same parameters as SearchSysCache. Per complaint from Marko Kreen. http://git.postgresql.org/pg/commitdiff/d4bf3c9c94305e692349fb6fe0c67e483b72ae87
  • Add GetForeignColumnOptions() to foreign.c, and add some documentation. GetForeignColumnOptions provides some abstraction for accessing column-specific FDW options, on a par with the access functions that were already provided here for other FDW-related information. Adjust file_fdw.c to use GetForeignColumnOptions instead of equivalent hand-rolled code. In addition, add some SGML documentation for the functions exported by foreign.c that are meant for use by FDW authors. (This is the fdw_helper portion of the proposed pgsql_fdw patch.) Hanada Shigeru, reviewed by KaiGai Kohei http://git.postgresql.org/pg/commitdiff/9088d1b96504717fd589ff7eeacc96b6d1c08ead
  • Fix indentation of \d footers for non-ASCII cases. Multi-line "Inherits:" and "Child tables:" footers were misindented when those strings' translations involved multibyte characters, because we were using strlen() instead of an appropriate display width measurement. In passing, avoid doing gettext() more than once per loop in these places. While at it, fix pg_wcswidth(), which has been entirely broken since about 8.2, but fortunately has been unused for the same length of time. Report and patch by Sergey Burladyan (bug #6480) http://git.postgresql.org/pg/commitdiff/1ed7f0e6b90a9b693895105a90d8b5b0eefbcd56
  • Improve estimation of IN/NOT IN by assuming array elements are distinct. In constructs such as "x IN (1,2,3,4)" and "x <> ALL(ARRAY[1,2,3,4])", we formerly always used a general-purpose assumption that the probability of success is independent for each comparison of "x" to an array element. But in real-world usage of these constructs, that's a pretty poor assumption; it's much saner to assume that the array elements are distinct and so the match probabilities are disjoint. Apply that assumption if the operator appears to behave as equality (for ANY) or inequality (for ALL). But fall back to the normal independent-probabilities calculation if this yields an impossible result, ie probability > 1 or < 0. We could protect ourselves against bad estimates even more by explicitly checking for equal array elements, but that is expensive and doesn't seem worthwhile: doing it would amount to optimizing for poorly-written queries at the expense of well-written ones. Daniele Varrazzo and Tom Lane, after a suggestion by Ants Aasma http://git.postgresql.org/pg/commitdiff/66a7e6bae98592d1d98d9ef589753f0e953c5828
  • Fix some issues with temp/transient tables in extension scripts. Phil Sorber reported that a rewriting ALTER TABLE within an extension update script failed, because it creates and then drops a placeholder table; the drop was being disallowed because the table was marked as an extension member. We could hack that specific case but it seems likely that there might be related cases now or in the future, so the most practical solution seems to be to create an exception to the general rule that extension member objects can only be dropped by dropping the owning extension. To wit: if the DROP is issued within the extension's own creation or update scripts, we'll allow it, implicitly performing an "ALTER EXTENSION DROP object" first. This will simplify cases such as extension downgrade scripts anyway. No docs change since we don't seem to have documented the idea that you would need ALTER EXTENSION DROP for such an action to begin with. Also, arrange for explicitly temporary tables to not get linked as extension members in the first place, and the same for the magic pg_temp_nnn schemas that are created to hold them. This prevents assorted unpleasant results if an extension script creates a temp table: the forced drop at session end would either fail or remove the entire extension, and neither of those outcomes is desirable. Note that this doesn't fix the ALTER TABLE scenario, since the placeholder table is not temp (unless the table being rewritten is). Back-patch to 9.1. http://git.postgresql.org/pg/commitdiff/08dd23cec7d6b5493c83848d7568495815eda5c6
  • Revise FDW planning API, again. Further reflection shows that a single callback isn't very workable if we desire to let FDWs generate multiple Paths, because that forces the FDW to do all work necessary to generate a valid Plan node for each Path. Instead split the former PlanForeignScan API into three steps: GetForeignRelSize, GetForeignPaths, GetForeignPlan. We had already bit the bullet of breaking the 9.1 FDW API for 9.2, so this shouldn't cause very much additional pain, and it's substantially more flexible for complex FDWs. Add an fdw_private field to RelOptInfo so that the new functions can save state there rather than possibly having to recalculate information two or three times. In addition, we'd not thought through what would be needed to allow an FDW to set up subexpressions of its choice for runtime execution. We could treat ForeignScan.fdw_private as an executable expression but that seems likely to break existing FDWs unnecessarily (in particular, it would restrict the set of node types allowable in fdw_private to those supported by expression_tree_walker). Instead, invent a separate field fdw_exprs which will receive the postprocessing appropriate for expression trees. (One field is enough since it can be a list of expressions; also, we assume the corresponding expression state tree(s) will be held within fdw_state, so we don't need to add anything to ForeignScanState.) Per review of Hanada Shigeru's pgsql_fdw patch. We may need to tweak this further as we continue to work on that patch, but to me it feels a lot closer to being right now. http://git.postgresql.org/pg/commitdiff/b14953932dfdda7d915b9e276a09df8458efeec8
  • Restructure SPGiST opclass interface API to support whole-index scans. The original API definition was incapable of supporting whole-index scans because there was no way to invoke leaf-value reconstruction without checking any qual conditions. Also, it was inefficient for multiple-qual-condition scans because value reconstruction got done over again for each qual condition, and because other internal work in the consistent functions likewise had to be done for each qual. To fix these issues, pass the whole scankey array to the opclass consistent functions, instead of only letting them see one item at a time. (Essentially, the loop over scankey entries is now inside the consistent functions not outside them. This makes the consistent functions a bit more complicated, but not unreasonably so.) In itself this commit does nothing except save a few cycles in multiple-qual-condition index scans, since we can't support whole-index scans on SPGiST indexes until nulls are included in the index. However, I consider this a must-fix for 9.2 because once we release it will get very much harder to change the opclass API definition. http://git.postgresql.org/pg/commitdiff/03e56f798e365763486b03a2630fbc3190ccd29a
  • Teach SPGiST to store nulls and do whole-index scans. This patch fixes the other major compatibility-breaking limitation of SPGiST, that it didn't store anything for null values of the indexed column, and so could not support whole-index scans or "x IS NULL" tests. The approach is to create a wholly separate search tree for the null entries, and use fixed "allTheSame" insertion and search rules when processing this tree, instead of calling the index opclass methods. This way the opclass methods do not need to worry about dealing with nulls. Catversion bump is for pg_am updates as well as the change in on-disk format of SPGiST indexes; there are some tweaks in SPGiST WAL records as well. Heavily rewritten version of a patch by Oleg Bartunov and Teodor Sigaev. (The original also stored nulls separately, but it reused GIN code to do so; which required undesirable compromises in the on-disk format, and would likely lead to bugs due to the GIN code being required to work in two very different contexts.) http://git.postgresql.org/pg/commitdiff/c6a11b89e48dfb47b305cea405924333dabc20b6
  • Fix documented type of t_infomask2. Per Koizumi Satoru http://git.postgresql.org/pg/commitdiff/1e4964478add0278a3f7ff685bc033f0f52625ad
  • Make parameter name consistent with syntax summary. Thomas Hunger http://git.postgresql.org/pg/commitdiff/8142166162efb193b1fece5cf32afb07fe24e17e
  • Make INSERT/UPDATE queries depend on their specific target columns. We have always created a whole-table dependency for the target relation, but that's not really good enough, as it doesn't prevent scenarios such as dropping an individual target column or altering its type. So we have to create an individual dependency for each target column, as well. Per report from Bill MacArthur of a rule containing UPDATE breaking after such an alteration. Note that this patch doesn't try to make such cases work, only to ensure that the attempted ALTER TABLE throws an error telling you it can't cope with adjusting the rule. This is a long-standing bug, but given the lack of prior reports I'm not going to risk back-patching it. A back-patch wouldn't do anything to fix existing rules' dependency lists, anyway. http://git.postgresql.org/pg/commitdiff/c6be1f43ab0551a95ec8ac77364e2f8558ae6345

Bruce Momjian a poussé :

Heikki Linnakangas a poussé :

  • Remove extra copies of LogwrtResult. This simplifies the code a little bit. The new rule is that to update XLogCtl->LogwrtResult, you must hold both WALWriteLock and info_lck, whereas before we had two copies, one that was protected by WALWriteLock and another protected by info_lck. The code that updates them was already holding both locks, so merging the two is trivial. The third copy, XLogCtl->Insert.LogwrtResult, was not totally redundant, it was used in AdvanceXLInsertBuffer to update the backend-local copy, before acquiring the info_lck to read the up-to-date value. But the value of that seems dubious; at best it's saving one spinlock acquisition per completed WAL page, which is not significant compared to all the other work involved. And in practice, it's probably not saving even that much. http://git.postgresql.org/pg/commitdiff/7714c6382941383514c0f1954ca831686ac4fcd2
  • Simplify the way changes to full_page_writes are logged. It's harmless to do full page writes even when not strictly necessary, so when turning full_page_writes on, we can set the global flag first, and then call XLogInsert. Likewise, when turning it off, we can write the WAL record first, and then clear the flag. This way XLogInsert doesn't need any special handling of the XLOG_FPW_CHANGE record type. XLogInsert is complicated enough already, so anything we can keep away from there is a good thing. Actually I don't think the atomicity of the shared memory flag matters, anyway, because we only write the XLOG_FPW_CHANGE at the end of recovery, when there are no concurrent WAL insertions going on. But might as well make it safe, in case we allow changing full_page_writes on the fly in the future. http://git.postgresql.org/pg/commitdiff/3b682df3260aa8e020201e4b6c5cbc31fe8ecb8e
  • Make the comments more clear on the fact that UpdateFullPageWrites() is not safe to call concurrently from multiple processes. http://git.postgresql.org/pg/commitdiff/e587e2e3e39ec54772905b8e0ac7155a03253934
  • Silence warning about unused variable, when building without assertions. http://git.postgresql.org/pg/commitdiff/d93f209f483f006534ae543667a1254b6fdec183
  • Update outdated comment. HeapTupleHeader.t_natts field doesn't exist anymore. Kevin Grittner http://git.postgresql.org/pg/commitdiff/342baf4ce61f06ad3898490dc5125579d9e6bd18

Robert Haas a poussé :

Tatsuo Ishii a poussé :

Michael Meskes a poussé :

Correctifs rejetés (à ce jour)

  • Pas de déception cette semaine :-)

Correctifs en attente

  • Pavel Stehule and Alvaro Herrera traded patches for the CHECK FUNCTION system.
  • Shigeru HANADA sent in four more revisions of the patch to add a PostgreSQL FDW.
  • Kyotaro HORIGUCHI and Marko Kreen traded patches to add a new method of storing tuples to libpq and use same to make dblink more efficient.
  • KaiGai Kohei and Yeb Havinga traded patches to add a new sepgsql.client_label GUC.
  • Dimitri Fontaine sent in three more revisions of the patch to add command triggers.
  • Tomas Vondra sent in two revisions of a patch to fix some regression test errors that appear in a Czech locale, cs_CZ.
  • Robert Haas sent in a patch to speed up the creation of error messages.
  • Bruce Momjian sent in two more revisions of a patch to fix the documentation for pg_upgrade --logfile.
  • Alexander Shulgin sent in two more revisions of a patch to support URI connection strings in libpq.
  • Pavel Stehule and Petr (PJMODOS) Jelinek traded patches to add CHECK TRIGGER and related functionality.
  • Fujii Masao sent in a patch to extend pg_stat_statements so that it reports the planning time.
  • Jaime Casanova and Robert Haas traded patches to extend pg_archivecleanup.
  • Jaime Casanova sent in a trimmed-down version of the patch to add GIN and SP-GiST support to pgstattuple.
  • Marti Raudsepp sent in a patch to optimize certain cases where IS DISTINCT FROM NULL elides to IS NOT NULL.
  • Robert Haas sent in a patch to add a pg_prewarm utility.
  • Fujii Masao sent in a patch to fix a bug in walsender which causes high CPU usage.
  • Antonin Houska sent in a WIP patch implementing some sub-cases of LATERAL for function calls.
  • Marti Raudsepp sent in another revision of the patch to refactor simplify_function.

par N Bougain le mardi 13 mars 2012 à 09h19

mercredi 7 mars 2012

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 4 mars 2012

Publication des mises à jour de sécurité : 9.1.3, 9.0.7, 8.4.11 et 8.3.18. Mettez à jour immédiatement si les vulnérabilités vous concernent, à la prochaine occasion sinon. http://www.postgresql.org/about/news/1377/

[ndt: annonce en français (fr)]

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en mars

PostgreSQL Local

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA.

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

(lien vers l'article original)

Revues de code

Correctifs appliqués

Tom Lane a poussé :

  • Remove arbitrary limitation on length of common name in SSL certificates. Both libpq and the backend would truncate a common name extracted from a certificate at 32 bytes. Replace that fixed-size buffer with dynamically allocated string so that there is no hard limit. While at it, remove the code for extracting peer_dn, which we weren't using for anything; and don't bother to store peer_cn longer than we need it in libpq. This limit was not so terribly unreasonable when the code was written, because we weren't using the result for anything critical, just logging it. But now that there are options for checking the common name against the server host name (in libpq) or using it as the user's name (in the server), this could result in undesirable failures. In the worst case it even seems possible to spoof a server name or user name, if the correct name is exactly 32 bytes and the attacker can persuade a trusted CA to issue a certificate in which that string is a prefix of the certificate's common name. (To exploit this for a server name, he'd also have to send the connection astray via phony DNS data or some such.) The case that this is a realistic security threat is a bit thin, but nonetheless we'll treat it as one. Back-patch to 8.4. Older releases contain the faulty code, but it's not a security problem because the common name wasn't used for anything interesting. Reported and patched by Heikki Linnakangas Security: CVE-2012-0867 http://git.postgresql.org/pg/commitdiff/077711c2e3e86384d19d833233bd35e05b921cfc
  • Convert newlines to spaces in names written in pg_dump comments. pg_dump was incautious about sanitizing object names that are emitted within SQL comments in its output script. A name containing a newline would at least render the script syntactically incorrect. Maliciously crafted object names could present a SQL injection risk when the script is reloaded. Reported by Heikki Linnakangas, patch by Robert Haas Security: CVE-2012-0868 http://git.postgresql.org/pg/commitdiff/89e0bac86dbca40dfc321926205f2a90d3da5437
  • Require execute permission on the trigger function for CREATE TRIGGER. This check was overlooked when we added function execute permissions to the system years ago. For an ordinary trigger function it's not a big deal, since trigger functions execute with the permissions of the table owner, so they couldn't do anything the user issuing the CREATE TRIGGER couldn't have done anyway. However, if a trigger function is SECURITY DEFINER, that is not the case. The lack of checking would allow another user to install it on his own table and then invoke it with, essentially, forged input data; which the trigger function is unlikely to realize, so it might do something undesirable, for instance insert false entries in an audit log table. Reported by Dinesh Kumar, patch by Robert Haas Security: CVE-2012-0866 http://git.postgresql.org/pg/commitdiff/891e6e7bfd9bb72687522af08c18689f795cb60a
  • Last-minute release note updates. Security: CVE-2012-0866, CVE-2012-0867, CVE-2012-0868 http://git.postgresql.org/pg/commitdiff/b2ce60703ab431a1d6c10f50587ea5f5e984af2e
  • Fix thinko in new match_join_clauses_to_index() logic. We don't need to constrain the other side of an indexable join clause to not be below an outer join; an example here is SELECT FROM t1 LEFT JOIN t2 ON t1.a = t2.b LEFT JOIN t3 ON t2.c = t3.d; We can consider an inner indexscan on t3.d using c = d as indexqual, even though t2.c is potentially nulled by a previous outer join. The comparable logic in orindxpath.c has always worked that way, but I was being overly cautious here. http://git.postgresql.org/pg/commitdiff/0140a11b9ba5b22e1e4807e178bca770d46c3e28
  • Move CRC tables to libpgport, and provide them in a separate include file. This makes it much more convenient to build tools for Postgres that are separately compiled and require a matching CRC implementation. To prevent multiple copies of the CRC polynomial tables being introduced into the postgres binaries, they are now included in the static library libpgport that is mainly meant for replacement system functions. That seems like a bit of a kludge, but there's no better place. This cleans up building of the tools pg_controldata and pg_resetxlog, which previously had to build their own copies of pg_crc.o. In the future, external programs that need access to the CRC tables can include the tables directly from the new header file pg_crc_tables.h. Daniel Farina, reviewed by Abhijit Menon-Sen and Tom Lane http://git.postgresql.org/pg/commitdiff/5c02a00d440b90ead12658ce6ec9f4eee95dd0a3
  • Fix MSVC builds for previous patch's addition of a src/port file. (And why in the world is this OBJS list not being scraped from the corresponding Makefile?) http://git.postgresql.org/pg/commitdiff/8cae5810ebaaabb54171d9953bdd9cc802f0d135
  • Simplify references to backslash-doubling in func.sgml. Several places were still written as though standard_conforming_strings didn't exist, much less be the default. Now that it is on by default, we can simplify the text and just insert occasional notes suggesting that you might have to think harder if it's turned off. Per discussion of a suggestion from Hannes Frederic Sowa. Back-patch to 9.1 where standard_conforming_strings was made the default. http://git.postgresql.org/pg/commitdiff/a5c1a1969dd838189e5cc936c15cb40e13fb6d68
  • Allow child-relation entries to be made in ec_has_const EquivalenceClasses. This fixes an oversight in commit 11cad29c91524aac1d0b61e0ea0357398ab79bf8, which introduced MergeAppend plans. Before that happened, we never particularly cared about the sort ordering of scans of inheritance child relations, since appending their outputs together would destroy any ordering anyway. But now it's important to be able to match child relation sort orderings to those of the surrounding query. The original coding of add_child_rel_equivalences skipped ec_has_const EquivalenceClasses, on the originally-correct grounds that adding child expressions to them was useless. The effect of this is that when a parent variable is equated to a constant, we can't recognize that index columns on the equivalent child variables are not sort-significant; that is, we can't recognize that a child index on, say, (x, y) is able to generate output in "ORDER BY y" order when there is a clause "WHERE x = constant". Adding child expressions to the (x, constant) EquivalenceClass fixes this, without any downside that I can see other than a few more planner cycles expended on such queries. Per recent gripe from Robert McGehee. Back-patch to 9.1 where MergeAppend was introduced. http://git.postgresql.org/pg/commitdiff/44634e474fcb9dcd92b16fe3a0fb1d8a91e69353
  • Collect and use element-frequency statistics for arrays. This patch improves selectivity estimation for the array <@, &&, and @> (containment and overlaps) operators. It enables collection of statistics about individual array element values by ANALYZE, and introduces operator-specific estimators that use these stats. In addition, ScalarArrayOpExpr constructs of the forms "const = ANY/ALL (array_column)" and "const <> ANY/ALL (array_column)" are estimated by treating them as variants of the containment operators. Since we still collect scalar-style stats about the array values as a whole, the pg_stats view is expanded to show both these stats and the array-style stats in separate columns. This creates an incompatible change in how stats for tsvector columns are displayed in pg_stats: the stats about lexemes are now displayed in the array-related columns instead of the original scalar-related columns. There are a few loose ends here, notably that it'd be nice to be able to suppress either the scalar-style stats or the array-element stats for columns for which they're not useful. But the patch is in good enough shape to commit for wider testing. Alexander Korotkov, reviewed by Noah Misch and Nathan Boley http://git.postgresql.org/pg/commitdiff/0e5e167aaea4ceb355a6e20eec96c4f7d05527ab
  • Improve histogram-filling loop in new compute_array_stats() code. Do "frac" arithmetic in int64 to prevent overflow with large statistics targets, and improve the comments so people have some chance of understanding how it works. Alexander Korotkov and Tom Lane http://git.postgresql.org/pg/commitdiff/4fb694aebc524f2085152d8c98a85e01ef6136f4
  • Remove useless "rough estimate" path from mcelem_array_contained_selec. The code in this function that tried to cope with a missing count histogram was quite ineffective for anything except a perfectly flat distribution. Furthermore, since we were already punting for missing MCELEM slot, it's rather useless to sweat over missing DECHIST: there are no cases where ANALYZE will create the first but not the second. So just simplify the code by punting rather than pretending we can do something useful. http://git.postgresql.org/pg/commitdiff/e2eed7891008cbf2b7d3868b3d77751b33ed09ad
  • Rewrite GiST support code for rangetypes. This patch installs significantly smarter penalty and picksplit functions for ranges, making GiST indexes for them smaller and faster to search. There is no on-disk format change, so no catversion bump, but you'd need to REINDEX to get the benefits for any existing index. Alexander Korotkov, reviewed by Jeff Davis http://git.postgresql.org/pg/commitdiff/80da9e68fdd70b796b3a7de3821589513596c0f7

Peter Eisentraut a poussé :

Alvaro Herrera a poussé :

  • ALTER TABLE: skip FK validation when it's safe to do so. We already skip rewriting the table in these cases, but we still force a whole table scan to validate the data. This can be skipped, and thus we can make the whole ALTER TABLE operation just do some catalog touches instead of scanning the table, when these two conditions hold: (a) Old and new pg_constraint.conpfeqop match exactly. This is actually stronger than needed; we could loosen things by way of operator families, but it'd require a lot more effort. (b) The functions, if any, implementing a cast from the foreign type to the primary opcintype are the same. For this purpose, we can consider a binary coercion equivalent to an exact type match. When the opcintype is polymorphic, require that the old and new foreign types match exactly. (Since ri_triggers.c does use the executor, the stronger check for polymorphic types is no mere future-proofing. However, no core type exercises its necessity.) Author: Noah Misch Committer's note: catalog version bumped due to change of the Constraint node. I can't actually find any way to have such a node in a stored rule, but given that we have "out" support for them, better be safe. http://git.postgresql.org/pg/commitdiff/cb3a7c2b95a28e57c56562d48d2a3aa5eeb7fa29
  • psql: when tab-completing, use quotes on file names that need them. psql backslash commands that deal with file or directory names require quotes around those that have spaces, single quotes, or backslashes. However, tab-completing such names does not provide said quotes, and is thus almost useless with them. This patch fixes the problem by having a wrapper function around rl_filename_completion_function that dequotes on input and quotes on output. This eases dealing with such names. Author: Noah Misch http://git.postgresql.org/pg/commitdiff/41e3c94cac0e68257126b2d264dc5e877e892490
  • Fix typo in comment. Haifeng Liu http://git.postgresql.org/pg/commitdiff/58e9f974dcfae7c4c445631afad47d80deb83160
  • Remove TOAST table from pg_database. The only toastable column now is datacl, but we don't really support long ACLs anyway. The TOAST table should have been removed when the pg_db_role_setting catalog was introduced in commit 2eda8dfb52ed9962920282d8384da8bb4c22514d, but I forgot to do that. Per -hackers discussion on March 2011. http://git.postgresql.org/pg/commitdiff/3433c6ba002f711a60352c3518f30cda73d06087

Heikki Linnakangas a poussé :

  • Correctly detect SSI conflicts of prepared transactions after crash. A prepared transaction can get new conflicts in and out after preparing, so we cannot rely on the in- and out-flags stored in the statefile at prepare- time. As a quick fix, make the conservative assumption that after a restart, all prepared transactions are considered to have both in- and out-conflicts. That can lead to unnecessary rollbacks after a crash, but that shouldn't be a big problem in practice; you don't want prepared transactions to hang around for a long time anyway. Dan Ports http://git.postgresql.org/pg/commitdiff/d6a7271958e61fe8029087a34483437292f41f6f
  • When a GiST page is split during index build, it might not have a buffer. Previously it was thought that it's impossible as the code stands, because insertions create buffers as tuples are cascaded downwards, and index split also creaters buffers eagerly for all halves. But the example from Jay Levitt demonstrates that it can happen, when the root page is split. It's in fact OK if the buffer doesn't exist, so we just need to remove the sanity check. In fact, we've been discussing the possibility of destroying empty buffers to conserve memory, which would render the sanity check completely useless anyway. Fix by Alexander Korotkov http://git.postgresql.org/pg/commitdiff/2502f45979fca76a6b19a07c98d7a41737a3dc7b

Magnus Hagander a poussé :

Andrew Dunstan a poussé :

Correctifs rejetés (à ce jour)

  • Pas de déception cette semaine :-)

Correctifs en attente

  • Dimitri Fontaine, Andres Freund and Thom Brown sent in parts of the grand unified patch to implement command triggers.
  • Marti Raudsepp sent in a patch to make heap_open()/heap_close() consistently, which kicked off a short discussion about an implicit behavior of heap_open() that things depend on, including third-party things.
  • Josh Kupershmidt sent in a patch to fix a misleading error message from connectMaintenanceDatabase().
  • Kyotaro HORIGUCHI sent in another revision of the patch to create in libpq and use in dblink, a more space-efficient tuple storage mechanism.
  • Robert Haas sent in a WIP patch to set a new buffer flag BM_BGWRITER_CLEANED to every buffer the background writer cleans.
  • Bruce Momjian sent in two revisions of a patch to clarify pg_upgrade --logfile along with its documentation.
  • Alexander Shulgin sent in another revision of the patch to add URI connection string support to libpq.
  • Alvaro Herrera and Pavel Stehule traded revisions of the patch to add CHECK FUNCTION functionality to PostgreSQL.
  • Peter Geoghegan sent in another revision of the patch to normalize pg_stat_statements based in part on a review by Daniel Farina.
  • Simon Riggs sent in four more revisions of the patch to add some hints to COPY for large bulk loads.
  • Shigeru HANADA sent in another revision of the patch to add a PostgreSQL FDW.
  • Marti Raudsepp sent in another revision of the patch to enable caching the results of stable expressions with constant arguments.
  • Gilles Darold sent in two patches which embody two different approaches to implement a pg_is_in_exclusive_backup() function.
  • Robert Haas sent in a patch to add sortsupport to TEXT types.
  • Yeb Havinga sent in another revision of the patch to add a GUC named sepgsql.client_label.
  • Simon Riggs sent in three more revisions of a patch to make TRUNCATE more MVCC-safe.
  • Daniele Varazzo sent in a patch to improves the array selectivity estimation for = ANY and <> ALL, hence for the IN/NOT IN operators.
  • Jeff Janes sent in a patch to fix some cases of how sort memory grows.

par N Bougain le mercredi 7 mars 2012 à 01h37

mardi 6 mars 2012

Jean-Paul Argudo

PostgreSQL usecase : site Le bon coin

Bonjour à tous,

Cela faisait plusieurs mois que je savais que le site leboincoin.fr fonctionnait principalement sous PostgreSQL.

Christophe Legendre m'a fait une fleur en m'autorisant à l'interviewer pour le compte du site PostgreSQLFr. J'ai donc saisi ma chance et suis allé le questionner sur son utilisation de PostgreSQL.

J'ai été assez impressionné à la fois par son installation et l'utilisation de PostgreSQL, mais aussi par sa maîtrise de PostgreSQL. Mais ce qui m'a fait le plus plaisir c'est probablement son accueil, sa gentillesse et le fait qu'il a passé tout le temps nécessaire avec moi pour rapporter à la communauté une vision claire et précise de ce que le site leboncoin.fr fait avec PostgreSQL.

Comme d'habitude, j'ai écrit l'interview et il l'a relue et validée. Ce procédé garanti au lecteur la véracité des propos. Christophe était en effet assez remonté contre un certain journalisme, qui, l'année dernière, lui a fait tenir des propos qu'il n'a jamais dit. En clair, on lui a demandé de dire que PostgreSQL ça n'était finalement pas si bien que ça, ce qu'il m'a démenti catégoriquement.

Bref. Je ne me prends pas pour un journaliste, je n'en ai ni la formation ni le talent. J'espère juste que ce "usecase" permettra à certains qui hésiteraient encore à passer leur site à "fort traffic" sous PostgreSQL.

Certains seront intrigués par la baie HP. Vous avez des informations à son sujet sur cette page et un PDF de présentation ici. Résolument une arme de performances massives :D

Ce usecase va être très prochainement traduit en anglais et envoyé au groupe Advocacy de PostgreSQL, dans le but d'être diffusé directement sur le site postgresql.org.

L'interview est à sa place, sur le site PostgreSQLFr !

Bonne lecture à tous !

par Jean-Paul le mardi 6 mars 2012 à 09h03

dimanche 4 mars 2012

Samuel Roze

Redmine (Ruby) `load_missing_constant’: Object is not missing constant Issue! (ArgumentError)

Lors de l’installation conjointe de Webistrano (voir Installer Webistrano avec nginx et PostgreSQL sous Gentoo) et de Redmine (voir Installer Redmine avec PostgreSQL) sur un même serveur, Redmine m’as quelque peut surpris on me sortant cette erreur:

/usr/lib64/ruby/gems/1.8/gems/activesupport-2.3.14/lib/active_support/dependencies.rb:443:in `load_missing_constant': Object is not missing constant Issue! (ArgumentError)
	from /usr/lib64/ruby/gems/1.8/gems/activesupport-2.3.14/lib/active_support/dependencies.rb:106:in `const_missing'
	from /usr/lib64/ruby/gems/1.8/gems/activesupport-2.3.14/lib/active_support/dependencies.rb:118:in `const_missing'
	from /usr/lib64/ruby/gems/1.8/gems/activesupport-2.3.14/lib/active_support/dependencies.rb:124:in `send'
	from /usr/lib64/ruby/gems/1.8/gems/activesupport-2.3.14/lib/active_support/dependencies.rb:124:in `const_missing'
	from /home/redmine/app/models/project.rb:40
	from /usr/lib64/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require'
	from /usr/lib64/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `require'
	from /usr/lib64/ruby/gems/1.8/gems/activesupport-2.3.14/lib/active_support/dependencies.rb:184:in `require'
	from /usr/lib64/ruby/gems/1.8/gems/activesupport-2.3.14/lib/active_support/dependencies.rb:291:in `require_or_load'
	from /usr/lib64/ruby/gems/1.8/gems/activesupport-2.3.14/lib/active_support/dependencies.rb:451:in `load_missing_constant'
	from /usr/lib64/ruby/gems/1.8/gems/activesupport-2.3.14/lib/active_support/dependencies.rb:106:in `const_missing'
	from /usr/lib64/ruby/gems/1.8/gems/activesupport-2.3.14/lib/active_support/dependencies.rb:118:in `const_missing'
	from /usr/lib64/ruby/gems/1.8/gems/activesupport-2.3.14/lib/active_support/dependencies.rb:463:in `load_missing_constant'
	from /usr/lib64/ruby/gems/1.8/gems/activesupport-2.3.14/lib/active_support/dependencies.rb:122:in `const_missing'
	from /home/redmine/app/models/principal.rb:22
	from /usr/lib64/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require'
	from /usr/lib64/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `require'
	from /usr/lib64/ruby/gems/1.8/gems/activesupport-2.3.14/lib/active_support/dependencies.rb:184:in `require'
	from /usr/lib64/ruby/gems/1.8/gems/activesupport-2.3.14/lib/active_support/dependencies.rb:291:in `require_or_load'
	from /usr/lib64/ruby/gems/1.8/gems/activesupport-2.3.14/lib/active_support/dependencies.rb:250:in `depend_on'
	from /usr/lib64/ruby/gems/1.8/gems/activesupport-2.3.14/lib/active_support/dependencies.rb:162:in `require_dependency'
	from /home/redmine/vendor/plugins/redmine-gitolite/init.rb:2:in `evaluate_init_rb'
	from /usr/lib64/ruby/gems/1.8/gems/rails-2.3.14/lib/rails/plugin.rb:158:in `evaluate_init_rb'
	from /usr/lib64/ruby/gems/1.8/gems/activesupport-2.3.14/lib/active_support/core_ext/kernel/reporting.rb:11:in `silence_warnings'
	from /usr/lib64/ruby/gems/1.8/gems/rails-2.3.14/lib/rails/plugin.rb:154:in `evaluate_init_rb'
	from /usr/lib64/ruby/gems/1.8/gems/rails-2.3.14/lib/rails/plugin.rb:48:in `load'
	from /home/redmine/config/../vendor/plugins/engines/lib/engines/plugin.rb:44:in `load'
	from /usr/lib64/ruby/gems/1.8/gems/rails-2.3.14/lib/rails/plugin/loader.rb:38:in `load_plugins'
	from /usr/lib64/ruby/gems/1.8/gems/rails-2.3.14/lib/rails/plugin/loader.rb:37:in `each'
	from /usr/lib64/ruby/gems/1.8/gems/rails-2.3.14/lib/rails/plugin/loader.rb:37:in `load_plugins'
	from /usr/lib64/ruby/gems/1.8/gems/rails-2.3.14/lib/initializer.rb:369:in `load_plugins'
	from /usr/lib64/ruby/gems/1.8/gems/rails-2.3.14/lib/initializer.rb:165:in `process'
	from /usr/lib64/ruby/gems/1.8/gems/rails-2.3.14/lib/initializer.rb:113:in `send'
	from /usr/lib64/ruby/gems/1.8/gems/rails-2.3.14/lib/initializer.rb:113:in `run'
	from /home/redmine/config/environment.rb:24
[...]

Après de nombreuses recherches, j’ai résolu le problème en ajoutant une petite ligne au fichier config/environnement.rb de Redmine:

  config.gem 'pg'

Après la ligne suivante:

  config.gem 'coderay', :version => '~>1.0.0'

:)

par Samuel ROZE le dimanche 4 mars 2012 à 12h41

Installer Webistrano avec nginx et PostgreSQL sous Gentoo

Webistrano est une application Web développée en Ruby On Rails permettant le déploiement automatisé de vos applications, qu’elles soient Web ou non. Ainsi, via le front-end Web, vous n’aurez qu’à appuyer sur un bouton pour que Webistrano récupère vos sources depuis un dépôt (svn, git, …) et depuis un répertoire et le déploie sur votre ferme de serveurs, en lançant par exemple les différents tests que vous aurez effectués. Si une erreur apparaît, Webistrano annule tous les changements et laisse l’application fonctionnelle comme elle l’était.

Pour installer Webistrano, vous avez besoin de Ruby On Rails, d’un serveur Web et d’une base de données. Je vais utiliser nginx comme serveur Web et PostgreSQL comme base de données. Je ne vais pas vous expliquer comment installer nginx et PostgreSQL, il y a déjà de très bons articles qui vous l’expliquent.

Installation du Ruby On Rails

Pour installer Ruby On Rails sur Gentoo, rien de plus simple, installer rudy puis le gem rails :

# emerge -av  dev-lang/ruby
# gem install rails --include-dependencies
# gem install bundler

Configuration de la base de données

Nous allons créer une nouvelle base de données pour Webistrano, ainsi qu’un nouvel utilisateur:

# su postgres
$ psql
postgres=# CREATE USER webistrano WITH PASSWORD '-VOTRE-MOT-DE-PASSE-';
CREATE ROLE
postgres=# CREATE DATABASE webistrano WITH OWNER webistrano;
CREATE DATABASE

Téléchargement et configuration du Webistrano

On va télécharger le tarball de la dernière version de Webistrano depuis le dépôt GitHub et décompressez le à l’endroit que vous souhaitez. Ici, le chemin de Webistrano sera /home/www/webistrano/.

useradd webistrano
mkdir -p /home/www/webistano/
wget https://github.com/peritor/webistrano/tarball/master -O /home/www/webistano.tgz
cd /home/www
tar -xzf webistano.tgz
mv peritor-webistrano-* webistrano
rm webistano.tgz

Ensuite, nous allons configurer Webistrano en éditant les fichiers de configuration mais avant il faut les créer à partir des modèles fournis:

cd /home/www/webistrano
cp config/webistrano_config.rb.sample config/webistrano_config.rb
cp config/database.yml.sample config/database.yml

Éditez donc le fichier /home/www/webistrano/config/webistrano_config.rb comme vous le souhaitez, notamment en ce qui concerne les adresses mail d’émission et de réception, ainsi que les paramètres SMTP.

Ensuite, éditez les paramètres de connexion à la base de données dans le fichier /home/www/webistrano/config/database.yml. Voici mon fichier de configuration pour utiliser PostgreSQL:

production:
  adapter: postgresql
  database: webistrano
  host: localhost
  username: webistrano
  password: -VOTRE-MOT-DE-PASSE-
  encoding: utf8
  schema_search_path: public

Complétez la base de données de PostgreSQL nécessaire à Webistrano grâce à l’outil rake:

# bundle install
# RAILS_ENV=production rake db:migrate

Il est possible que vous ayez à mettre à jour rake:

# gem update rake --include-dependencies

Personnellement, j’ai eu une erreur avec rake me disant qu’il fallait installer le gem activerecord-postgresql-adapter. Même en installant le gem pg, postgres ou postgres-pr, rien ne changeait. En ajoutant gem "pg" dans le fichier Gemfile et en lançant bundle install, ça a marché!

Configuration de nginx

Pour faire fonctionner l’application Ruby On Rails avec nginx, nous allons utiliser le gem thin. Pour cela, il vous suffit de l’installer, comme vous pourrez le lire dans cet article.

Créez un fichier /etc/thin/webistrano.yml, dans lequel vous mettez ceci:

pid: tmp/pids/thin-webistrano.pid
group: webistrano
wait: 30
timeout: 30
log: log/thin-webistrano.log
max_conns: 1024
require: []
 
environment: production
max_persistent_conns: 512
servers: 4
daemonize: true
user: webistrano
socket: /tmp/thin-webistano.sock
chdir: /home/www/webistrano

Créez la configuration de votre domaine pour nginx, afin d’utiliser les 4 sockets unix de thin. Voici un fichier de configuration pour le nom de domaine webistrano.example.com avec la configuration de thin ci-dessus.

# Upstream Ruby process cluster for load balancing
upstream thin_webistrano_cluster {
    server unix:/tmp/thin-webistano.0.sock;
    server unix:/tmp/thin-webistano.1.sock;
    server unix:/tmp/thin-webistano.2.sock;
    server unix:/tmp/thin-webistano.3.sock;
}
 
server {
    listen       80;
    server_name  webistrano.example.com;
 
    access_log  /var/log/nginx/webistrano.example.com.access.log;
    error_log   /var/log/nginx/webistrano.example.com.error.log;
 
    include proxy.include;
    root /home/www/webistrano;
    proxy_redirect off;
 
    # Send sensitive stuff via https
    #rewrite ^/login(.*) https://webistrano.example.com$request_uri permanent;
    #rewrite ^/my/account(.*) https://webistrano.example.com$request_uri permanent;
    #rewrite ^/my/password(.*) https://webistrano.example.com$request_uri permanent;
    #rewrite ^/admin(.*) https://webistrano.example.com$request_uri permanent;
 
    location / {
        try_files $uri/index.html $uri.html $uri @cluster;
    }
 
    location @cluster {
        proxy_pass http://thin_webistrano_cluster;
    }
}

Il suffit maintenant de redemarrer thin et nginx pour prendre en compte toutes ces modifications:

# /etc/init.d/thin restart
# /etc/init.d/nginx restart

Vous pouvez maintenant accéder à Webistrano à l’adresse choisie (dans l’exemple, webistrano.example.com) avec les identifiants par défaut qui sont admin et admin.

par Samuel ROZE le dimanche 4 mars 2012 à 11h24

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 26 février 2012

Publication des mises à jours mineures 9.1.3, 9.0.7, 8.4.11 et 8.3.18 imminentes. Préparez-vous ! [ndt: mises à jour disponibles (fr)]

Le PGDay DC 2012 est programmé pour le 30 mars : http://pgday.bwpug.org

Le PGDay Austin 2012 est programmé pour le 28 mars : http://www.austinpug.org/events/50962652/

Offres d'emplois autour de PostgreSQL en février

PostgreSQL Local

  • Le PGDay NYC aura lieu le 2 avril 2012 au Lighthouse International à New-York : http://pgday.nycpug.org
  • La PGCon 2012 sera tenue à l'Université d'Ottawa, les 17 et 18 mai 2012. Elle sera précédée par deux jours de tutoriels les 15 & 16 mai 2012 : http://www.pgcon.org/2012/
  • Le PGDay France aura lieu à Lyon, le 7 juin 2012 : http://www.pgday.fr

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA.

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

(lien vers l'article original)

Revues de code

Correctifs appliqués

Andrew Dunstan a poussé :

Tom Lane a poussé :

  • Don't reject threaded Python on FreeBSD. According to Chris Rees, this has worked for awhile, and the current FreeBSD port is removing the test anyway. http://git.postgresql.org/pg/commitdiff/c0efc2c2ab416b805ba5ccea621d7198a3f3330f
  • Don't clear btpo_cycleid during _bt_vacuum_one_page. When "vacuuming" a single btree page by removing LP_DEAD tuples, we are not actually within a vacuum operation, but rather in an ordinary insertion process that could well be running concurrently with a vacuum. So clearing the cycleid is incorrect, and could cause the concurrent vacuum to miss removing tuples that it needs to remove. This is a longstanding bug introduced by commit e6284649b9e30372b3990107a082bc7520325676 of 2006-07-25. I believe it explains Maxim Boguk's recent report of index corruption, and probably some other previously unexplained reports. In 9.0 and up this is a one-line fix; before that we need to introduce a flag to tell _bt_delitems what to do. http://git.postgresql.org/pg/commitdiff/593a9631a7947ab95903e87e24786d7e469cc988
  • Cosmetic cleanup for commit a760893dbda9934e287789d54bbd3c4ca3914ce0. Mostly, fixing overlooked comments. http://git.postgresql.org/pg/commitdiff/9789c99d01e7e4460b77c29b77d177f86c45a273
  • Draft release notes for 9.1.3, 9.0.7, 8.4.11, 8.3.18. http://git.postgresql.org/pg/commitdiff/dd2954963b4da9f6e8ea759f19ca5eb0cf79010f
  • Allow MinGW builds to use standardly-named OpenSSL libraries. In the Fedora variant of MinGW, the openssl libraries have their normal names, not libeay32 and libssleay32. Adjust configure probes to allow that, per bug #6486. Tomasz Ostrowski http://git.postgresql.org/pg/commitdiff/74e29162a4f0ec0ad1c7224b8be936d1f9a51f7e
  • Stamp 9.1.3, 9.0.7, 8.4.11, 8.3.18. http://git.postgresql.org/pg/commitdiff/64c47e4542910ebbfb494bec3f8abf8733113394
  • Fix the general case of quantified regex back-references. Cases where a back-reference is part of a larger subexpression that is quantified have never worked in Spencer's regex engine, because he used a compile-time transformation that neglected the need to check the back-reference match in iterations before the last one. (That was okay for capturing parens, and we still do it if the regex has *only* capturing parens ... but it's not okay for backrefs.) To make this work properly, we have to add an "iteration" node type to the regex engine's vocabulary of sub-regex nodes. Since this is a moderately large change with a fair risk of introducing new bugs of its own, apply to HEAD only, even though it's a fix for a longstanding bug. http://git.postgresql.org/pg/commitdiff/173e29aa5deefd9e71c183583ba37805c8102a72
  • Avoid repeated creation/freeing of per-subre DFAs during regex search. In nested sub-regex trees, lower-level nodes created DFAs and then destroyed them again before exiting, which is a bit dumb considering that the recursive search is likely to call those nodes again later. Instead cache each created DFA until the end of pg_regexec(). This is basically a space for time tradeoff, in that it might increase the maximum memory usage. However, in most regex patterns there are not all that many subre nodes, so not that many DFAs --- and in any case, the peak usage occurs when reaching the bottom recursion level, and except for alternation cases that's going to be the same anyway. http://git.postgresql.org/pg/commitdiff/587359479acbbdc95c8e37da40707e37097423f5
  • Merge dissect() into cdissect() to remove a pile of near-duplicate code. The "uncomplicated" case isn't materially less complicated than the full case, certainly not enough so to justify duplicating nearly 500 lines of code. The only extra work being done in the full path is zaptreesubs, which is very cheap compared to everything else being done here, and besides that I'm less than convinced that it's not needed in some cases even without backrefs. http://git.postgresql.org/pg/commitdiff/4dd78bf37aa29d04b3f358b08c4a2fa43cf828e7
  • Remove useless "retry memory" logic within regex engine. Apparently some primordial version of Spencer's engine needed cdissect() and child functions to be able to continue matching from a previous position when re-called. That is dead code, though, since trivial inspection shows that cdissect can never be entered without having previously done zapmem which resets the relevant retry counter. I have also verified experimentally that no case in the Tcl regression tests reaches cdissect with a nonzero retry value. Accordingly, remove that logic. This doesn't really save any noticeable number of cycles in itself, but it is one step towards making dissect() and cdissect() equivalent, which will allow removing hundreds of lines of near-duplicated code. Since struct subre's "retry" field is no longer particularly related to any kind of retry, rename it to "id". As of this commit it's only used for identifying a subre node in debug printouts, so you might think we should get rid of the field entirely; but I have a plan for another use. http://git.postgresql.org/pg/commitdiff/3cbfe485e44d055b9e6a27e47069729375059f8c
  • Fix some more bugs in GIN's WAL replay logic. In commit 4016bdef8aded77b4903c457050622a5a1815c16 I fixed a bunch of ginxlog.c bugs having to do with not handling XLogReadBuffer failures correctly. However, in ginRedoUpdateMetapage and ginRedoDeleteListPages, I unaccountably thought that failure to read the metapage would be impossible and just put in an elog(PANIC) call. This is of course wrong: failure is exactly what will happen if the index got dropped (or rebuilt) between creation of the WAL record and the crash we're trying to recover from. I believe this explains Nicholas Wilson's recent report of these errors getting reached. Also, fix memory leak in forgetIncompleteSplit. This wasn't of much concern when the code was written, but in a long-running standby server page split records could be expected to accumulate indefinitely. Back-patch to 8.4 --- before that, GIN didn't have a metapage. http://git.postgresql.org/pg/commitdiff/1b630751d0ffef4c856bfe382889d0d187eca404

Peter Eisentraut a poussé :

Magnus Hagander a poussé :

Simon Riggs a poussé :

Alvaro Herrera a poussé :

Bruce Momjian a poussé :

Robert Haas a poussé :

Correctifs rejetés (à ce jour)

  • Pas de déception cette semaine :-)

Correctifs en attente

  • Alexander Korotkov sent in another revision of the patch to fix some slowness and bugs in inserting cubes.
  • Kyotaro HORIGUCHI and Marko Kreen traded patches to create a new libpq tuple store and speed up dblink using same.
  • Etsuro Fujita and Shigeru HANADA traded versions of a patch to add a PostgreSQL FDW.
  • Gabriele Bartolini sent in another revision of the patch to allow array elements to reference keys in another table.
  • Dimitri Fontaine sent in another revision of the patch to collect and display accumulated AUTOVACUUM cost.
  • Alexander Korotkov sent in a patch to fix incorrect behaviour when using a GiST index on points.
  • Jan Urbanski sent in another revision of the patch to fix potential reference miscounts and segfaults in plpython.c.
  • Greg Smith sent in a patch to add a pg_test_timing tool for EXPLAIN ANALYZE overhead.
  • Simon Riggs sent in another revision of the patch to assess the overall level of freelist lwlock contention.
  • Greg Smith and Ants Aasma traded revisions of a patch to add timing of buffer I/O requests.
  • Peter Geoghegan sent in another revision of the patch to publish checkpoint timing and sync files summary data to pg_stat_bgwriter.
  • Timothy Garnett sent in a patch to make it possible to ask pg_dump to dump tables in clustered index order.
  • Joachim Wieland sent in another revision of the patch to make it possible to use pg_dump in parallel.
  • Daniel Farina sent in another revision of the patch to move CRC tables to a separate include file and libpg.
  • KaiGai Kohei sent in another revision of the patch to add a sepgsql_setcon() function to SE-pgsql.
  • Peter Eisentraut sent in a patch to fix an issue with incompatible pointer types for different revisions of zlib.
  • Alex Shulgin sent in another revision of a patch to add URL support to libpq.
  • Daniel Farina sent in another revision of a patch inteneded to re-normalize pg_stat_statements.
  • Peter Eisentraut sent in another revision of the patch to correct misleading errors on CREATE TABLE ... LIKE.
  • Simon Riggs sent in a patch to speed up COPY for the case of adding committed rows.
  • Dimitri Fontaine sent in three more revisions of the patch to implement command triggers.
  • Sergey Burladyan sent in a patch to fix a bug in psql's indent for inherited tables names with UTF-8
  • Noah Misch sent in a patch to cut down the noise in psql when there is an error in a multi-command string.
  • Jeff Janes sent in a patch to reproduce and fix an issue he's been seeing around the "moving more work outside the WALInsertLock" patch.
  • Magnus Hagander sent in a patch to fix xlog location arithmetic.
  • Simon Riggs sent in a patch to fix certain contentions in CLOG.
  • Magnus Hagander sent in a patch to create a new stylesheet, assuming the docs are built locally.

par N Bougain le dimanche 4 mars 2012 à 03h03

lundi 27 février 2012

Actualités PostgreSQL.fr

Mises à jour mineures de PostgreSQL : 9.1.3, 9.0.7, 8.4.11, 8.3.18

Le projet PostgreSQL sort aujourd'hui des mises à jour de sécurité pour toutes les branches actives du SGBD PostgreSQL. Ces mises à jour correspondent aux versions 9.1.3, 9.0.7, 8.4.11, 8.3.18.

Les utilisateurs de pg_dump, de certificats SSL pour la validation ou de triggers utilisant une fonction en SECURITY DEFINER doivent mettre à jour immédiatement leur installation. Il est fortement conseillé à tous les autres administrateurs de mettre à jour leur version de PostgreSQL lors du prochain arrêt planifié de la base. Des détails sur les correctifs de sécurité suivent ci-dessous.

Les fonctionnalités affectées par les corrections de cette mise à jour incluent : la réplication binaire et le serveurs en Hot Standby, les index GIN, les requêtes CTE, les wrappers de données distantes, les langages PL/pgsql et PL/python, le type de données inet, les modules intarray, pgcrypto et pg_upgrade ainsi que les outils pg_restore et pg_dump. Les utilisateurs de ces fonctionnalités doivent installer la mise à jour dès que possible.

Cette mise à jour contient 46 correctifs pour la version 9.1. Les versions plus anciennes ont moins de correctifs. Parmi les correctifs :

  • Correction d'une corruption des index Btree lors d'insertions en parallèle à un VACUUM ;
  • Retour à la normale après des erreurs survenant lors du rejeu d'un DROP TABLESPACE
  • Correction de la mise à zéro transitoire du cache disque de PostgreSQL lors du rejeu des journaux de transactions
  • Correction du postmaster pour tenter un redémarrage après un crash d'un serveur Hot Standby
  • Correction d'un cas particulier dans le nettoyage des transactions SSI
  • Mise à jour des droits par colonne des autorisations, et non pas seulement des droits par table, lors du changement du propriétaire d'une table
  • Correction de la gestion des sous-plans d'une requête CTE en écriture lors de la vérification du READ COMMITTED
  • Correction des échecs causant le message « could not find plan for CTE »
  • Correction de l'erreur «type de nœud non supporté» causé par la clause COLLATE dans une expression INSERT
  • Correction d'un crash lors de problèmes sur la suppression de fichiers de données après une validation (COMMIT)
  • Correction de la fuite de mémoire récemment introduite dans le traitement des types de données inet / cidr
  • Correction de l'estimation des coûts de la gestion des colonnes dans une clause IN (...) avec un index GIN
  • Correction des fuites de mémoire lors des conversions d'entrées/sorties dans PL/pgsql
  • Amélioration de pg_upgrade permettant le renommage de la bibliothèque partagée PL/python (affectant les mises à jour vers la 9.1)

Comme pour les autres versions mineures, il n'est pas nécessaire de sauvegarder et recharger les bases de données. Il n'est pas utile non plus d'utiliser pg_upgrade. Pour appliquer cette mise à jour, arrêtez PostgreSQL, mettez à jour les exécutables et redémarrez PostgreSQL. Puis réalisez les étapes post-mise-à-jour une fois le serveur redémarré.

Cette mise à jour inclut les trois correctifs de sécurité suivants:

  • CVE-2012-0866: Permissions on a function called by a trigger are not checked. (Les droits sur une fonction appelée par un trigger ne sont pas vérifiés)

Ce correctif empêche les utilisateurs de définir des triggers qui exécutent des fonctions pour lesquelles l'utilisateur n'a pas le droit EXECUTE.

CREATE TRIGGER ne parvenait pas à vérifier tous les droits sur la fonction trigger à appeler. Un utilisateur sans droit pouvait attacher une fonction trigger à une table dont il est propriétaire et faire en sorte qu'elle soit exécutée avec les données de son choix. Habituellement, cela exécute la fonction avec les droits du propriétaire de la table et ne devrait pas donner de droits supplémentaires. Néanmoins, si une fonction trigger est marquée SECURITY DEFINER, une escalade des droits est possible.

  • CVE-2012-0867: SSL certificate name checks are truncated to 32 characters, allowing connection spoofing under some circumstances. (Les vérifications du nom du certificat SSL sont tronquées à 32 caractères, permettant un « spoofing » de connexion dans certaines circonstances)

Cela corrige le tronquage du nom commun SSL, qui permettait à un attaquant le détournement d'une connexion SSL dans des circonstances exceptionnelles.

Lors de l'utilisation de certificats SSL, les clients et serveurs peuvent être configurés pour vérifier le nom d'hôte de l'autre partie par rapport au nom commun compris dans le certificat que cette partie présente. Néanmoins, le nom extrait à partir du certificat était tronqué par erreur à 32 caractères. Habituellement, cela résulte en un échec de la vérification mais si le nom actuel de l'hôte fait exactement 32 caractères, il pourrait, en principe, être forgé. Le risque est très mince et un attaquant aurait toujours besoin de réaliser d'autres étapes, en dehors de PostgreSQL, pour réussir à exploiter cela.

  • CVE-2012-0868: Line breaks in object names can be exploited to execute code when loading a pg_dump file. (Les sauts de lignes dans le nom des objets peuvent être exploités pour exécuter du code lors du chargement d'un fichier pg_dump)

Ce correctif supprime les caractères \n et \r des commentaires compris dans la sauvegarde.

pg_dump copiait les noms des objets dans des commentaires du script SQL sans les sécuriser. Un nom d'objet contenant un retour à la ligne suivi d'une commande SQL résultait en l'exécution de cette commande lors de la restauration. Si le script SQL contenant la sauvegarde est exécuté, la commande s'exécute avec les droits de la personne qui restaure la sauvegarde, souvent un superutilisateur.

Toutes les versions supportées de PostgreSQL sont affectées. Les notes de version de chaque branche contiennent une liste complète des modifications avec de nombreux détails.

Téléchargez les nouvelles versions maintenant sur :

La version originale de cette annonce est disponible ici : http://www.postgresql.org/about/new...

Merci à Guillaume pour la traduction !

par daamien le lundi 27 février 2012 à 23h21

vendredi 24 février 2012

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 19 février 2012

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en février

PostgreSQL Local

  • Le PGDay NYC aura lieu le 2 avril 2012 au Lighthouse International à New-York : http://pgday.nycpug.org
  • La PGCon 2012 sera tenue à l'Université d'Ottawa, les 17 et 18 mai 2012. Elle sera précédée par deux jours de tutoriels les 15 & 16 mai 2012 : http://www.pgcon.org/2012/
  • Le PGDay France aura lieu à Lyon, le 7 juin 2012 : http://www.pgday.fr

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA.

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

(lien vers l'article original)

Revues de code

Correctifs appliqués

Heikki Linnakangas a poussé :

Andrew Dunstan a poussé :

Michael Meskes a poussé :

Bruce Momjian a poussé :

Robert Haas a poussé :

Tom Lane a poussé :

  • Preserve column names in the execution-time tupledesc for a RowExpr. The hstore and json datatypes both have record-conversion functions that pay attention to column names in the composite values they're handed. We used to not worry about inserting correct field names into tuple descriptors generated at runtime, but given these examples it seems useful to do so. Observe the nicer-looking results in the regression tests whose results changed. catversion bump because there is a subtle change in requirements for stored rule parsetrees: RowExprs from ROW() constructs now have to include field names. Andrew Dunstan and Tom Lane http://git.postgresql.org/pg/commitdiff/398f70ec070fe60151584eaa448f04708aa77892
  • Run a portal's cleanup hook immediately when pushing it to FAILED state. This extends the changes of commit 6252c4f9e201f619e5eebda12fa867acd4e4200e so that we run the cleanup hook earlier for failure cases as well as success cases. As before, the point is to avoid an assertion failure from an Assert I added in commit a874fe7b4c890d1fe3455215a83ca777867beadd, which was meant to check that no user-written code can be called during portal cleanup. This fixes a case reported by Pavan Deolasee in which the Assert could be triggered during backend exit (see the new regression test case), and also prevents the possibility that the cleanup hook is run after portions of the portal's state have already been recycled. That doesn't really matter in current usage, but it foreseeably could matter in the future. Back-patch to 9.1 where the Assert in question was added. http://git.postgresql.org/pg/commitdiff/4bfe68dfab009ce8fcaea79dc0832eadf3380051
  • Improve statistics estimation to make some use of DISTINCT in sub-queries. Formerly, we just punted when trying to estimate stats for variables coming out of sub-queries using DISTINCT, on the grounds that whatever stats we might have for underlying table columns would be inapplicable. But if the sub-query has only one DISTINCT column, we can consider its output variable as being unique, which is useful information all by itself. The scope of this improvement is pretty narrow, but it costs nearly nothing, so we might as well do it. Per discussion with Andres Freund. This patch differs from the draft I submitted yesterday in updating various comments about vardata.isunique (to reflect its extended meaning) and in tweaking the interaction with security_barrier views. There does not seem to be a reason why we can't use this sort of knowledge even when the sub-query is such a view. http://git.postgresql.org/pg/commitdiff/4767bc8ff2edc1258cf4d8a83155d4cedd724231
  • Fix longstanding error in contrib/intarray's int[] & int[] operator. The array intersection code would give wrong results if the first entry of the correct output array would be "1". (I think only this value could be at risk, since the previous word would always be a lower-bound entry with that fixed value.) Problem spotted by Julien Rouhaud, initial patch by Guillaume Lelarge, cosmetic improvements by me. http://git.postgresql.org/pg/commitdiff/06d9afa6f93ec08a45da4de7afd97bbf16738739
  • Sync regex code with Tcl 8.5.11. Sync our regex code with upstream changes since last time we did this, which was Tcl 8.5.0 (see commit df1e965e12cdd48c11057ee6e15346ee2b8b02f5). There are no functional changes here; the main point is just to lay down a commit-log marker that somebody has looked at this recently, and to do what we can to keep the two codebases comparable. http://git.postgresql.org/pg/commitdiff/08fd6ff37f71485e2fc04bc6ce07d2a483c36702
  • Update expected/collate.linux.utf8.out for recent plpgsql changes. This file was missed in commit 4c6cedd1b014abf2046886a9a92e10e18f0d658e. http://git.postgresql.org/pg/commitdiff/759c95c45b65a5220976c85e6f03323975c2b276
  • Create the beginnings of internals documentation for the regex code. Create src/backend/regex/README to hold an implementation overview of the regex package, and fill it in with some preliminary notes about the code's DFA/NFA processing and colormap management. Much more to do there of course. Also, improve some code comments around the colormap and cvec code. No functional changes except to add one missing assert. http://git.postgresql.org/pg/commitdiff/27af91438b68f46f4015853b6f75c6f5c3a8650c
  • Add caching of ctype.h/wctype.h results in regc_locale.c. While this doesn't save a huge amount of runtime, it still seems worth doing, especially since I realized that the data copying I did in my first draft was quite unnecessary. In this version, once we have the results cached, getting them back for re-use is really very cheap. Also, remove the hard-wired limitation to not consider wctype.h results for character codes above 255. It turns out that we can't push the limit as far up as I'd originally hoped, because the regex colormap code is not efficient enough to cope very well with character classes containing many thousand letters, which a Unicode locale is entirely capable of producing. Still, we can push it up to U+7FF (which I chose as the limit of 2-byte UTF8 characters), which will at least make Eastern Europeans happy pending a better solution. Thus, this commit resolves the specific complaint in bug #6457, but not the more general issue that letters of non-western alphabets are mostly not recognized as matching [[:alpha:]]. http://git.postgresql.org/pg/commitdiff/e00f68e49c148851187136d3278b7e9afa370537
  • Fix regex back-references that are directly quantified with *. The syntax "\n*", that is a backref with a * quantifier directly applied to it, has never worked correctly in Spencer's library. This has been an open bug in the Tcl bug tracker since 2005: https://sourceforge.net/tracker/index.php?func=detail&aid=1115587&group_id=10894&atid=110894 The core of the problem is in parseqatom(), which first changes "\n*" to "\n+|" and then applies repeat() to the NFA representing the backref atom. repeat() thinks that any arc leading into its "rp" argument is part of the sub-NFA to be repeated. Unfortunately, since parseqatom() already created the arc that was intended to represent the empty bypass around "\n+", this arc gets moved too, so that it now leads into the state loop created by repeat(). Thus, what was supposed to be an "empty" bypass gets turned into something that represents zero or more repetitions of the NFA representing the backref atom. In the original example, in place of ^([bc])\1*$ we now have something that acts like ^([bc])(\1+|[bc]*)$ At runtime, the branch involving the actual backref fails, as it's supposed to, but then the other branch succeeds anyway. We could no doubt fix this by some rearrangement of the operations in parseqatom(), but that code is plenty ugly already, and what's more the whole business of converting "x*" to "x+|" probably needs to go away to fix another problem I'll mention in a moment. Instead, this patch suppresses the *-conversion when the target is a simple backref atom, leaving the case of m == 0 to be handled at runtime. This makes the patch in regcomp.c a one-liner, at the cost of having to tweak cbrdissect() a little. In the event I went a bit further than that and rewrote cbrdissect() to check all the string-length-related conditions before it starts comparing characters. It seems a bit stupid to possibly iterate through many copies of an n-character backreference, only to fail at the end because the target string's length isn't a multiple of n --- we could have found that out before starting. The existing coding could only be a win if integer division is hugely expensive compared to character comparison, but I don't know of any modern machine where that might be true. This does not fix all the problems with quantified back-references. In particular, the code is still broken for back-references that appear within a larger expression that is quantified (so that direct insertion of the quantification limits into the BACKREF node doesn't apply). I think fixing that will take some major surgery on the NFA code, specifically introducing an explicit iteration node type instead of trying to transform iteration into concatenation of modified regexps. Back-patch to all supported branches. In HEAD, also add a regression test case for this. (It may seem a bit silly to create a regression test file for just one test case; but I'm expecting that we will soon import a whole bunch of regex regression tests from Tcl, so might as well create the infrastructure now.) http://git.postgresql.org/pg/commitdiff/5223f96d92fd6fb6fcf260da9f9cb111831f0b37

Correctifs rejetés (à ce jour)

  • Pas de déception cette semaine :-)

Correctifs en attente

  • Marko Kreen and Kyotaro HORIGUCHI traded patches to create, and use in dblink, a new tuple storage for libpq.
  • Etsuro Fujita and Shigeru HANADA traded patches to make a PostgreSQL FDW.
  • Alvaro Herrera sent in another revision of the patch to create a special lock type for foreign keys.
  • Heikki Linnakangas sent in two more revisions of a patch to scale xlog insertion.
  • Dimitri Fontaine sent in a patch to fix an issue with DROP EXTENSION.
  • Chetan Suttraway sent in another revision of the patch to optimize referential integrity checks.
  • Jaime Casanova sent in a flock of patches which: adds gin and spgist support to pgstattuple and makes pgstattuple use a ring buffer when reading tables or indexes, adds the relation_free_space function to pgstattuple, and adds a stats_target parameter to the relation_free_space() function.
  • Dimitri Fontaine sent in four more revisions of the patch to add command triggers.
  • Alexander Korotkov sent in two more revision of the patch to make some speed improvements on inserting and indexing cubes.
  • Dan Ports sent in a patch to fix a possible incompatibility between prepared transactions and SSI.
  • Peter Eisentraut sent in another revision of the patch to control the location of server-side SSL files via a new GUC.
  • MauMau sent in a patch to fix a bug in windows debug builds where the postmaster would always crash.
  • Simon Riggs sent in three more revisions of the patch to add page checksums.
  • Kevin Grittner sent in another revision of the patch to (re-)run GUC check hooks on RESET.
  • Dan Scales sent in a patch implementing a new option for wal_sync_method intended to improve performance.
  • Peter Geoghegan sent in another revision of the patch to normalize pg_stat_statements.
  • Robert Haas sent in another revision of the patch to display autovacuum accumulated cost.
  • Robert Haas sent in another revision of the patch to simulate clog contention.
  • Peter Eisentraut sent in a patch to make pg_regress set the application name rather than leaving it as psql.
  • Jan Urbanski sent in a patch to add PL/Python execution contexts.
  • Jan Urbanski sent in a patch to fix some reference miscounts and segfaults in PL/Python.
  • Simon Riggs sent in a patch intended to reduce the frequency of bgwriter wakeups.
  • Pavel Stehule sent in a patch to add tab completion for functions to psql.
  • Pavel Stehule sent in a patch to add tab completion for CREATE OR REPLACE FUNCTION in psql.

par N Bougain le vendredi 24 février 2012 à 00h50

mardi 21 février 2012

Damien Clochard

PG Session #3 : Bilan et perspectives

La 3eme conférence "PG Session" s'est achevée il y a quelques jours et je vous propose ici un rapide compte-rende de cette journée à la fois riche et passionnante.

Tout d'abord je tiens à remercier les orateurs qui nous ont fait l'honneur de participer à cette conférence, notamment : Michael Paquier (NTT Data Intellilink / Japon), Simon Riggs (2nd Quadrant / Angleterre) et Ludovic Levesque (Fotolia / France). La qualité et la diversité de leurs interventions est la démonstration même de la richesse l'écosystème PostgreSQL !

Sur la base des différents thèmes abordés, j'ai tiré les 4 pistes de réflexion suivantes :

A/ La réplication interne de PostgreSQL est simple et mature. Le mécanisme Hot Standby répond à la grande majorité des besoins des entreprises : Haute-Disponibilité, Répartition des lectures, PRA... Avec une configuration intuitive et bien documenté, le Hot Standby s'est imposé en moins de 2 ans comme le système de réplication le plus populaire pour PostgreSQL. En bref si vous êtes à la recherche d'une solution de redondance pour vos données, mettez en place tout d'abord en place un Hot Standby vous serez rarement déçus !

B/ Pour les besoins plus spécifiques telles que les réplications croisées, les clusters hétérogènes ou les réplication master-master, il existe un panel de solutions sophistiquées : Slony, Londiste, Bucardo sont autant de logiciels libres qui vous permettront de déployez des architectures complexes avec PostgreSQL

C/ La scalabilité en écriture est la prochaine frontière à atteindre. L'objectif déclaré est de concurrencer directement Oracle RAC. Plusieurs projets sont déjà très avancés dans cette voie : On peut notamment citer le prometteur Postgres-XC qui devrait sortir en version 1.0 dans les semaines qui viennent. N'hésitez pas à tester ce projet et à renvoyer du feedback aux auteurs.

D/ Quelque soit la solution de réplication choisie, il est essentiel de ne pas négliger le temps de maintenance et le coût de la montée en compétence de vos équipes. Cela tombe bien car au delà de l'aspect open-source, PostgreSQL propose un véritable changement de paradigme économique : le coût total de possession (TCO) d'un systèmes de réplications . Avec un SGBD propriétaire, les licences seront le poste de dépenses le plus important. Avec PostgreSQL, le coût d'un cluster de réplication est le coût est avant tout humain : formation, conseil, support . En clair : plutôt que d'enrichir des milliardaires californiens , votre budget "réplication" sera convertit en services (audit de préconisation, support 24h/24, transfert de compétences, etc.) via une des nombreuses société qui compose l'écosystème PostgreSQL.

De notre propre expérience chez Dalibo : la plupart de clients qui sont passés d'Oracle à PostgreSQL sont séduits par cette approche à la fois pragmatique et humaine, aussi bien niveau technique que commercial !

J'en profite d'ailleurs pour annoncer que la prochaine Session PostgreSQL se tiendra cet automne et abordera le sujet de la transition d'Oracle vers PostgreSQL : outil de migration, remplacer Data Guard, remplacer OEM, etc.... Si vous cherchez une alternative à Oracle, rendez- vous en septembre !

D'ici là, abonnez-vous à notre liste de diffusion pour être tenu informés des dernières nouvelles de la communauté PostgreSQL....

Vous pouvez également consulter les slides de orateurs sur le site des PG Sessions : http://www.postgresql-sessions.org/...

par damien le mardi 21 février 2012 à 21h19

dimanche 19 février 2012

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 12 février 2012

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en février

PostgreSQL Local

  • Le PGDay NYC aura lieu le 2 avril 2012 au Lighthouse International à New-York : http://pgday.nycpug.org
  • La PGCon 2012 sera tenue à l'Université d'Ottawa, les 17 et 18 mai 2012. Elle sera précédée par deux jours de tutoriels les 15 & 16 mai 2012 : http://www.pgcon.org/2012/
  • Le PGDay France aura lieu à Lyon, le 7 juin 2012 : http://www.pgday.fr

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA.

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

(lien vers l'article original)

Revues de code

Correctifs appliqués

Alvaro Herrera a poussé :

Robert Haas a poussé :

Tom Lane a poussé :

  • Add locking around WAL-replay modification of shared-memory variables. Originally, most of this code assumed that no Postgres backends could be running concurrently with it, and so no locking could be needed. That assumption fails in Hot Standby. While it's still true that Hot Standby backends should never change values like nextXid, they can examine them, and consistency is important in some cases such as when computing a snapshot. Therefore, prudence requires that WAL replay code obtain the relevant locks when modifying such variables, even though it can examine them without taking a lock. We were following that coding rule in some places but not all. This commit applies the coding rule uniformly to all updates of ShmemVariableCache and MultiXactState fields; a search of the replay routines did not find any other cases that seemed to be at risk. In addition, this commit fixes a longstanding thinko in replay of NEXTOID and checkpoint records: we tried to advance nextOid only if it was behind the value in the WAL record, but the comparison would draw the wrong conclusion if OID wraparound had occurred since the previous value. Better to just unconditionally assign the new value, since OID assignment shouldn't be happening during replay anyway. The additional locking seems to be more in the nature of future-proofing than fixing any live bug, so I am not going to back-patch it. The NEXTOID fix will be back-patched separately. http://git.postgresql.org/pg/commitdiff/c6d76d7c82ebebb7210029f7382c0ebe2c558bca
  • Avoid problems with OID wraparound during WAL replay. Fix a longstanding thinko in replay of NEXTOID and checkpoint records: we tried to advance nextOid only if it was behind the value in the WAL record, but the comparison would draw the wrong conclusion if OID wraparound had occurred since the previous value. Better to just unconditionally assign the new value, since OID assignment shouldn't be happening during replay anyway. The consequences of a failure to update nextOid would be pretty minimal, since we have long had the code set up to obtain another OID and try again if the generated value is already in use. But in the worst case there could be significant performance glitches while such loops iterate through many already-used OIDs before finding a free one. The odds of a wraparound happening during WAL replay would be small in a crash-recovery scenario, and the length of any ensuing OID-assignment stall quite limited anyway. But neither of these statements hold true for a replication slave that follows a WAL stream for a long period; its behavior upon going live could be almost unboundedly bad. Hence it seems worth back-patching this fix into all supported branches. Already fixed in HEAD in commit c6d76d7c82ebebb7210029f7382c0ebe2c558bca. http://git.postgresql.org/pg/commitdiff/f1b8a84dec30b44e6a0b306f95961f5426cb8368
  • Avoid throwing ERROR during WAL replay of DROP TABLESPACE. Although we will not even issue an XLOG_TBLSPC_DROP WAL record unless removal of the tablespace's directories succeeds, that does not guarantee that the same operation will succeed during WAL replay. Foreseeable reasons for it to fail include temp files created in the tablespace by Hot Standby backends, wrong directory permissions on a standby server, etc etc. The original coding threw ERROR if replay failed to remove the directories, but that is a serious overreaction. Throwing an error aborts recovery, and worse means that manual intervention will be needed to get the database to start again, since otherwise the same error will recur on subsequent attempts to replay the same WAL record. And the consequence of failing to remove the directories is only that some probably-small amount of disk space is wasted, so it hardly seems justified to throw an error. Accordingly, arrange to report such failures as LOG messages and keep going when a failure occurs during replay. Back-patch to 9.0 where Hot Standby was introduced. In principle such problems can occur in earlier releases, but Hot Standby increases the odds of trouble significantly. Given the lack of field reports of such issues, I'm satisfied with patching back as far as the patch applies easily. http://git.postgresql.org/pg/commitdiff/5fc78efcec01fd5e857278556ad4312ae94ecc58
  • Fix postmaster to attempt restart after a hot-standby crash. The postmaster was coded to treat any unexpected exit of the startup process (i.e., the WAL replay process) as a catastrophic crash, and not try to restart it. This was OK so long as the startup process could not have any sibling postmaster children. However, if a hot-standby backend crashes, we SIGQUIT the startup process along with everything else, and the resulting exit is hardly "unexpected". Treating it as such meant we failed to restart a standby server after any child crash at all, not only a crash of the WAL replay process as intended. Adjust that. Back-patch to 9.0 where hot standby was introduced. http://git.postgresql.org/pg/commitdiff/442231d7f71764b8c628044e7ce2225f9aa43b67
  • Mark some more I/O-conversion-invoking functions as stable not volatile. When written, textanycat, anytextcat, quote_literal, and quote_nullable were marked volatile, because they could invoke arbitrary type-specific output functions as part of casting their anyelement arguments to text. Since then, we have defined a project policy that I/O functions must not be volatile, as per commit aab353a60b95aadc00f81da0c6d99bde696c4b75. So these functions can safely be downgraded to stable. Most of the time this makes no difference since they'll get inlined anyway, but as noted by Andrew Dunstan, there are cases where the volatile marking prevents optimizations that the planner does before function inlining. (I think I might have overlooked these functions in the earlier commit on the grounds that inlining would make it moot, but not so --- tgl) This change results in a change in the expected output of the json regression tests, because the planner can now flatten a sub-select that it failed to before. The old output is preferable, but getting that back will require some as-yet-unfinished work on RowExpr handling. Marti Raudsepp http://git.postgresql.org/pg/commitdiff/3db6524fe63f0598dcb2b307bb422bc126f2b15d
  • Support min/max index optimizations on boolean columns. Since bool_and() is equivalent to min(), and bool_or() to max(), we might as well let them be index-optimized in the same way. The practical value of this is debatable at best, but it seems nearly cost-free to enable it. Code-wise, we need only adjust the entries in pg_aggregate. There is a measurable planning speed penalty for a query involving one of these aggregates, but it is only a few percent in simple cases, so that seems acceptable. Marti Raudsepp, reviewed by Abhijit Menon-Sen http://git.postgresql.org/pg/commitdiff/cbba55d6d792b55f6b448a31fc14aef84510967c
  • Check misplaced window functions before checking aggregate/group by sanity. If somebody puts a window function in WHERE, we should complain about that in so many words. The previous coding tended to complain about the window function's arguments instead, which is likely to be misleading to users who are unclear on the semantics of window functions; as seen for example in bug #6440 from Matyas Novak. Just another example of how "add new code at the end" is frequently a bad heuristic. http://git.postgresql.org/pg/commitdiff/cb7c84fae8a6780d836687aa2c9655eb936ebd25
  • Fix up dumping conditions for extension configuration tables. Various filters that were meant to prevent dumping of table data were not being applied to extension config tables, notably --exclude-table-data and --no-unlogged-table-data. We also would bogusly try to dump data from views, sequences, or foreign tables, should an extension try to claim they were config tables. Fix all that, and refactor/redocument to try to make this a bit less fragile. This reverts the implementation, though not the feature, of commit 7b070e896ca835318c90b02c830a5c4844413b64, which had broken config-table dumping altogether :-(. It is still the case that the code will dump config-table data even if --schema is specified. That behavior was intentional, as per the comments in getExtensionMembership, so I think it requires some more discussion before we change it. http://git.postgresql.org/pg/commitdiff/d77354eaec53ed469a6f2444813ff3a4fd9d7a48
  • Throw error sooner for unlogged GiST indexes. Throwing an error only after we've built the main index fork is pretty unfriendly when the table already contains data. Per gripe from Jay Levitt. http://git.postgresql.org/pg/commitdiff/331bf6712c71a1c110bc52423eede8b4bac221a1
  • Add ORDER BY to a query to prevent occasional regression test failures. Per buildfarm, we sometimes get row-ordering variations in the output. This also makes this query look more like numerous other ones in the same test file. http://git.postgresql.org/pg/commitdiff/d06e2d200562837afa18058937f20460a3ea526a
  • Fix pg_dump for better handling of inherited columns. Revise pg_dump's handling of inherited columns, which was last looked at seriously in 2001, to eliminate several misbehaviors associated with inherited default expressions and NOT NULL flags. In particular make sure that a column is printed in a child table's CREATE TABLE command if and only if it has attislocal = true; the former behavior would sometimes cause a column to become marked attislocal when it was not so marked in the source database. Also, stop relying on textual comparison of default expressions to decide if they're inherited; instead, don't use default-expression inheritance at all, but just install the default explicitly at each level of the hierarchy. This fixes the search-path-related misbehavior recently exhibited by Chester Young, and also removes some dubious assumptions about the order in which ALTER TABLE SET DEFAULT commands would be executed. Back-patch to all supported branches. http://git.postgresql.org/pg/commitdiff/00bc96bd2b6646c73a073aa91dc68ed4718cf5f3
  • Fix brain fade in previous pg_dump patch. In pre-7.3 databases, pg_attribute.attislocal doesn't exist. The easiest way to make sure the new inheritance logic behaves sanely is to assume it's TRUE, not FALSE. This will result in printing child columns even when they're not really needed. We could work harder at trying to reconstruct a value for attislocal, but there is little evidence that anyone still cares about dumping from such old versions, so just do the minimum necessary to have a valid dump. I had this correct in the original draft of the patch, but for some unaccountable reason decided it wasn't necessary to change the value. Testing against an old server shows otherwise... http://git.postgresql.org/pg/commitdiff/97dc3c8a147c01da38570e4be7b4979af918dca2
  • Fix oversight in pg_dump's handling of extension configuration tables. If an extension has not been selected to be dumped (perhaps because of a --schema or --table switch), the contents of its configuration tables surely should not get dumped either. Per gripe from Hubert Depesz Lubaczewski. http://git.postgresql.org/pg/commitdiff/59de132f9a578ae5d2909228484a61309df986e0
  • Fix I/O-conversion-related memory leaks in plpgsql. Datatype I/O functions are allowed to leak memory in CurrentMemoryContext, since they are generally called in short-lived contexts. However, plpgsql calls such functions for purposes of type conversion, and was calling them in its procedure context. Therefore, any leaked memory would not be recovered until the end of the plpgsql function. If such a conversion was done within a loop, quite a bit of memory could get consumed. Fix by calling such functions in the transient "eval_econtext", and adjust other logic to match. Back-patch to all supported versions. Andres Freund, Jan Urbański, Tom Lane http://git.postgresql.org/pg/commitdiff/58a9596ed4a509467e1781b433ff9c65a4e5b5ce

Michael Meskes a poussé :

Heikki Linnakangas a poussé :

Peter Eisentraut a poussé :

Bruce Momjian a poussé :

Magnus Hagander a poussé :

Correctifs rejetés (à ce jour)

  • Pas de déception cette semaine :-)

Correctifs en attente

  • Marco Nenciarini sent in another revision of the patch to allow the elements of arrays to be enforced as foreign keys.
  • Fujii Masao sent in two revisions of a patch to fix incorrect handling of the timeout in pg_receivexlog.
  • Marko Kreen sent in another revision of the patch to create a new tuple storage in libpq and use same to make dblink more efficient in some cases.
  • Chetan Suttraway sent in another revision of the patch to implement SPI_gettypemod().
  • Fujii Masao sent in a patch to fix an issue where pg_basebackup -x stream from the standby gets stuck.
  • Chetan Suttraway sent in a patch to prevent the specification of conflicting transaction read/write options.
  • Euler Taveira de Oliveira sent in another revision of the patch to do xlog location arithmetic.
  • Shigeru HANADA sent in two more revisions of the patch to add a PostgreSQL FDW.
  • Marti Raudsepp sent in another revision of a patch to remove an optimization barrier involving the volatility of text-any concatenation.
  • Alvaro Herrera and Alex Hunsaker traded patches to fix an issue with missing keywords in make.
  • Peter Eisentraut sent in a patch to fix some of the missing things in psql's SELECT tab completion.
  • Marti Raudsepp sent in a patch to make TRUNCATE more MVCC-safe.
  • Peter Geoghegan sent in another revision of the patch for fast-path ordering, b-tree index creation time.
  • Alex Hunsaker sent in a patch to fix a bug in PL/Perl in databases encoded as SQL_ASCII.
  • Shigeru HANADA sent in another revision of the patch to collect statistics on CSV files attached via FDW.
  • Jean-Baptiste Quenot sent in a patch to fix an issue with PL/Python's handling of result metadata.
  • Kevin Grittner sent in a patch to ensure that if a GUC has a check function, it is run on a RESET just like it is on a SET, to make sure that the resulting value is valid to set within the context.
  • Andrew Dunstan sent in two revisions of a patch to fix a case where auto_explain can produce invalid JSON.
  • Jeff Janes sent in a WIP patch to to set XLP_FIRST_IS_CONTRECORD, this being part of the continuing effort to move more work outside WALInsertLock.
  • Vik Reykja sent in a patch to optimize referential integrity checks.

par N Bougain le dimanche 19 février 2012 à 13h28

vendredi 10 février 2012

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 5 février 2012

Le PGDay France aura lieu à Lyon, le 7 juin 2012 : http://www.pgday.fr

Le PGDay NYC aura lieu le 2 avril 2012 au Lighthouse International à New-York : http://pgday.nycpug.org

Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en février

PostgreSQL Local

  • La cinquième conférence annuelle "Prague PostgreSQL Developers Day", organisée pas le CSPUG (PUG Tchèque & Slovaque), aura lieu le 9 février 2012 à Prague.
  • Le PGDay NYC aura lieu le 2 avril 2012 au Lighthouse International à New-York : http://pgday.nycpug.org
  • La PGCon 2012 sera tenue à l'Université d'Ottawa, les 17 et 18 mai 2012. Elle sera précédée par deux jours de tutoriels les 15 & 16 mai 2012 : http://www.pgcon.org/2012/
  • Le PGDay France aura lieu à Lyon, le 7 juin 2012 : http://www.pgday.fr

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA.

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

(lien vers l'article original)

Revues de code

Correctifs appliqués

Heikki Linnakangas a poussé :

  • Accept a non-existent value in "ALTER USER/DATABASE SET ..." command. When default_text_search_config, default_tablespace, or temp_tablespaces setting is set per-user or per-database, with an "ALTER USER/DATABASE SET ..." statement, don't throw an error if the text search configuration or tablespace does not exist. In case of text search configuration, even if it doesn't exist in the current database, it might exist in another database, where the setting is intended to have its effect. This behavior is now the same as search_path's. Tablespaces are cluster-wide, so the same argument doesn't hold for tablespaces, but there's a problem with pg_dumpall: it dumps "ALTER USER SET ..." statements before the "CREATE TABLESPACE" statements. Arguably that's pg_dumpall's fault - it should dump the statements in such an order that the tablespace is created first and then the "ALTER USER SET default_tablespace ..." statements after that - but it seems better to be consistent with search_path and default_text_search_config anyway. Besides, you could still create a dump that throws an error, by creating the tablespace, running "ALTER USER SET default_tablespace", then dropping the tablespace and running pg_dumpall on that. Backpatch to all supported versions. http://git.postgresql.org/pg/commitdiff/a5782570402988205b3a42ed40f7bc157eac21fc
  • Make group commit more effective. When a backend needs to flush the WAL, and someone else is already flushing the WAL, wait until it releases the WALInsertLock and check if we still need to do the flush or if the other backend already did the work for us, before acquiring WALInsertLock. This helps group commit, because when the WAL flush finishes, all the backends that were waiting for it can be woken up in one go, and the can all concurrently observe that they're done, rather than waking them up one by one in a cascading fashion. This is based on a new LWLock function, LWLockWaitUntilFree(), which has peculiar semantics. If the lock is immediately free, it grabs the lock and returns true. If it's not free, it waits until it is released, but then returns false without grabbing the lock. This is used in XLogFlush(), so that when the lock is acquired, the backend flushes the WAL, but if it's not, the backend first checks the current flush location before retrying. Original patch and benchmarking by Peter Geoghegan and Simon Riggs, although this patch as committed ended up being very different from that. http://git.postgresql.org/pg/commitdiff/9b38d46d9f5517dab67dda1dd0459683fc9cda9f
  • Fix bug in the new wait-until-lwlock-is-free mechanism. If there was a wait-until-free process in the head of the wait queue, followed by an exclusive locker, the exclusive locker was not be woken up as it should. http://git.postgresql.org/pg/commitdiff/82d4b262d9614958da38003cdc9d56915d9bcda0
  • Print function signature, not just name, in PL/pgSQL error messages. This makes it unambiguous which function the message is coming from, if you have overloaded functions. Pavel Stehule, reviewed by Abhijit Menon-Sen. http://git.postgresql.org/pg/commitdiff/4c6cedd1b014abf2046886a9a92e10e18f0d658e

Simon Riggs a poussé :

Robert Haas a poussé :

Peter Eisentraut a poussé :

Tom Lane a poussé :

  • Code review for plpgsql fn_signature patch. Don't quote the output of format_procedure(); it's already quoted quite enough. Remove the fn_name field, which was now just dead weight. Fix remaining expected-output files. http://git.postgresql.org/pg/commitdiff/bef47331b6a1c0d081179a7fc9b82dd7e1862394
  • Try to be more consistent about accepting denormalized float8 numbers. On some platforms, strtod() reports ERANGE for a denormalized value (ie, one that can be represented as distinct from zero, but is too small to have full precision). On others, it doesn't. It seems better to try to accept these values consistently, so add a test to see if the result value indicates a true out-of-range condition. This should be okay per Single Unix Spec. On machines where the underlying math isn't IEEE standard, the behavior for such small numbers may not be very consistent, but then it wouldn't be anyway. Marti Raudsepp, after a proposal by Jeroen Vermeulen http://git.postgresql.org/pg/commitdiff/c318aeed84438619fc6b8c647def1730a110f04b
  • Add some regression test cases for denormalized float8 input. This was submitted with the previous patch, but I'm committing it separately to ease backing it out if these results prove too unportable. Marti Raudsepp, after a proposal by Jeroen Vermeulen http://git.postgresql.org/pg/commitdiff/500cf66d5522b39ddfdc26b309f8b5b0e385f42e
  • Revert "Add some regression test cases for denormalized float8 input." This reverts commit 500cf66d5522b39ddfdc26b309f8b5b0e385f42e. As was more or less expected, a small minority of platforms won't accept denormalized input even with the recent changes. It doesn't seem especially helpful to test this if we're going to have to provide an alternate expected-file to allow failure. http://git.postgresql.org/pg/commitdiff/342b83fdca6af04d86e761f4d8d79e5d598688cd
  • Allow SQL-language functions to reference parameters by name. Matthew Draper, reviewed by Hitoshi Harada http://git.postgresql.org/pg/commitdiff/9bff0780cf5be2193a5bad0d3df2dbe143085264
  • Improve comment. http://git.postgresql.org/pg/commitdiff/ee68a44106fa89b8efb2f21b71c3fcafaaf48851
  • Add missing Assert and fix inaccurate elog message in standby_redo(). All other WAL redo routines either call RestoreBkpBlocks() or Assert that they haven't been passed any backup blocks. Make this one do likewise. Also, fix incorrect routine name in its failure message. http://git.postgresql.org/pg/commitdiff/2af72cefeaa15e27277d327783fdec2748d9b758
  • Fix transient clobbering of shared buffers during WAL replay. RestoreBkpBlocks was in the habit of zeroing and refilling the target buffer; which was perfectly safe when the code was written, but is unsafe during Hot Standby operation. The reason is that we have coding rules that allow backends to continue accessing a tuple in a heap relation while holding only a pin on its buffer. Such a backend could see transiently zeroed data, if WAL replay had occasion to change other data on the page. This has been shown to be the cause of bug #6425 from Duncan Rance (who deserves kudos for developing a sufficiently-reproducible test case) as well as Bridget Frey's re-report of bug #6200. It most likely explains the original report as well, though we don't yet have confirmation of that. To fix, change the code so that only bytes that are supposed to change will change, even transiently. This actually saves cycles in RestoreBkpBlocks, since it's not writing the same bytes twice. Also fix seq_redo, which has the same disease, though it has to work a bit harder to meet the requirement. So far as I can tell, no other WAL replay routines have this type of bug. In particular, the index-related replay routines, which would certainly be broken if they had to meet the same standard, are not at risk because we do not have coding rules that allow access to an index page when not holding a buffer lock on it. Back-patch to 9.0 where Hot Standby was added. http://git.postgresql.org/pg/commitdiff/17118825b8164aac6d337b58cf66b17637c66a49

Alvaro Herrera a poussé :

Andrew Dunstan a poussé :

Michael Meskes a poussé :

Bruce Momjian a poussé :

Correctifs rejetés (à ce jour)

  • Pas de déception cette semaine :-)

Correctifs en attente

  • Alexander Korotkov sent in another revision of the patch to add GiST indexing for range types, per review by Jeff Davis.
  • Kyotaro HORIGUCHI and Marko Kreen traded patches to speed dblink by creating and using a new libpq tuple storage method.
  • Gilles Darold sent in a patch to implement pg_is_in_backup().
  • Alvaro Herrera sent in another revision of the FOREIGN KEY locks patch.
  • Robert Haas and Simon Riggs traded patches to implement DROP INDEX CONCURRENTLY.
  • Andrew Dunstan sent in a patch to fix an issue where pg_dump -s could dump data if it came from an extension.
  • Shigeru HANADA sent in another revision of the patch to implement a PostgreSQL FDW.
  • Heikki Linnakangas sent in another revision of the patch to scale xlog insertion.
  • Peter Eisentraut sent in another revision of the patch to enable renaming constraints in an ALTER TABLE statement.
  • Chetan Suttraway sent in a patch to implement SPI_gettypemod().
  • Simon Riggs sent in a patch refactoring log_newpage.
  • Alvaro Herrera sent in a patch to fix an issue where heap_tuple_read could get a false positive.
  • Alvaro Herrera sent in a patch to make options in pg_hba.conf be case-insensitive.
  • Oleg Bartunov sent in a patch to support NULLs in SP-GiST.
  • Robert Haas sent in another revision of the patch to allow EXPLAIN ANALYZE not to include timing information.
  • Marti Raudsepp sent in another revision of the patch to cache stable expressions with constant arguments.
  • KaiGai Kohei sent in another revision of the patch to unify the permissions checks on DROP.
  • KaiGai Kohei sent in another revision of the patch to add a sepgsql.client_label GUC.
  • Thomas Munro sent in a patch to add an optional SKIP LOCKED DATA to SELECT ... FOR (UPDATE | SHARE ).
  • Jeff Janes sent in another revision of the patch to use less memory during sorting.
  • Jeff Davis sent in a patch to fix an issue where it was possible for things to happen between an initdb call and its actually getting written out to disk.
  • Simon Riggs sent in another revision of the patch to fix an issue in slot_deform_tuple.
  • Jan Urbanski sent in a patch to fix a memory leak in PL/PythonU's datum->dict transformation.
  • Jan Urbanski sent in a patch to fix a memory leak in PL/PythonU's datum->string transformation.

par N Bougain le vendredi 10 février 2012 à 01h42

jeudi 2 février 2012

Actualités PostgreSQL.fr

Nouvelles hebdomadaires de PostgreSQL - 29 janvier 2012

Il y aura une série de conférences et événements dédiée à PostgreSQL au FOSDEM, à Bruxelles les 4 & 5 février prochains, en plus des conférenciers de la communauté PostgreSQL inscrits dans la série principale : http://fosdem.org/2012/

L'appel à conférenciers pour la PGCon a été étendu jusqu'au 31 janvier 2012 : http://www.pgcon.org/2012/papers.php

Le PGDay de New-York aura lieu le 2 avril 2012 au Lighthouse International : http://pgday.nycpug.org

Le PGDay français est programmé le 7 juin 2012 à Lyon : http://www.pgday.fr

Offres d'emplois autour de PostgreSQL en janvier

PostgreSQL Local

  • La cinquième conférence annuelle "Prague PostgreSQL Developers Day", organisée pas le CSPUG (PUG Tchèque & Slovaque), aura lieu le 9 février 2012 à Prague.
  • La PGCon 2012 sera tenue à l'Université d'Ottawa, les 17 et 18 mai 2012. Elle sera précédée par deux jours de tutoriels les 15 & 16 mai 2012 : http://www.pgcon.org/2012/

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA.

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

(lien vers l'article original)

Correctifs appliqués

Revues de code

Correctifs appliqués

Simon Riggs a poussé :

Robert Haas a poussé :

Bruce Momjian a poussé :

Tom Lane a poussé :

  • Suppress possibly-uninitialized-variable warning seen with older gcc versions. http://git.postgresql.org/pg/commitdiff/beef89567e3a0e3e1c753754a474065c5ad632fe
  • Suppress variable-clobbered-by-longjmp warning seen with older gcc versions. http://git.postgresql.org/pg/commitdiff/f26c9896b3da1b6aa5c560c4743f22a2a6c84919
  • Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments. http://git.postgresql.org/pg/commitdiff/e2fa76d80ba571d4de8992de6386536867250474
  • Undo 8.4-era lobotomization of subquery pullup rules. After the planner was fixed to convert some IN/EXISTS subqueries into semijoins or antijoins, we had to prevent it from doing that in some cases where the plans risked getting much worse. The reason the plans got worse was that in the unoptimized implementation, subqueries could reference parameters from the outer query at any join level, and so full table scans could be avoided even if they were one or more levels of join below where the semi/anti join would be. Now that we have sufficient mechanism in the planner to handle such cases properly, it should no longer be necessary to play dumb here. This reverts commits 07b9936a0f10d746e5076239813a5e938f2f16be and cd1f0d04bf06938c0ee5728fc8424d62bcf2eef3. The latter was a stopgap fix that wasn't really sufficiently analyzed at the time. Rather than just restricting ourselves to cases where the new join can be stacked on the right-hand input, we should also consider whether it can be stacked on the left-hand input. http://git.postgresql.org/pg/commitdiff/0816fad6eebddb8f1f0e21635e46625815d690b9
  • Fix error detection in contrib/pgcrypto's encrypt_iv() and decrypt_iv(). Due to oversights, the encrypt_iv() and decrypt_iv() functions failed to report certain types of invalid-input errors, and would instead return random garbage values. Marko Kreen, per report from Stefan Kaltenbrunner http://git.postgresql.org/pg/commitdiff/a8b4b84360e5ea145d12d372305b50f6774b0366
  • Fix handling of data-modifying CTE subplans in EvalPlanQual. We can't just skip initializing such subplans, because the referencing CTE node will expect to find the subplan available when it initializes. That in turn means that ExecInitModifyTable must allow the case (which actually it needed to do anyway, since there's no guarantee that ModifyTable is exactly at the top of the CTE plan tree). So move the complaint about not being allowed in EvalPlanQual mode to execution instead of initialization. Testing turned up yet another problem, which is that we'd try to re-initialize the result relation's index list, leading to leaks and dangling pointers. Per report from Phil Sorber. Back-patch to 9.1 where data-modifying CTEs were introduced. http://git.postgresql.org/pg/commitdiff/7c1719bc68ec1c347e7c80c3735bf3373e765f35
  • Add simple tests of EvalPlanQual using the isolationtester infrastructure. Much more could be done here, but at least now we have *some* automated test coverage of that mechanism. In particular this tests the writable-CTE case reported by Phil Sorber. In passing, remove isolationtester's arbitrary restriction on the number of steps in a permutation list. I used this so that a single spec file could be used to run several related test scenarios, but there are other possible reasons to want a step series that's not exactly a permutation. Improve documentation and fix a couple other nits as well. http://git.postgresql.org/pg/commitdiff/759d9d67695783f6d04a85aba383a41c5382548c
  • Fix handling of init_plans list in inheritance_planner(). Formerly we passed an empty list to each per-child-table invocation of grouping_planner, and then merged the results into the global list. However, that fails if there's a CTE attached to the statement, because create_ctescan_plan uses the list to find the plan referenced by a CTE reference; so it was unable to find any CTEs attached to the outer UPDATE or DELETE. But there's no real reason not to use the same list throughout the process, and doing so is simpler and faster anyway. Per report from Josh Berkus of "could not find plan for CTE" failures. Back-patch to 9.1 where we added support for WITH attached to UPDATE or DELETE. Add some regression test cases, too. http://git.postgresql.org/pg/commitdiff/4ec6581c0cdddfda767641f535116ee9a0412149
  • Update statement about sorting of character-string data. The sort order is no longer fixed at database creation time, but can be controlled via COLLATE. Noted by Thomas Kellerer. http://git.postgresql.org/pg/commitdiff/17d3233e1bfd9fbe856f032a542f2c595e79ca01
  • Add caution about multiple unique indexes breaking plpgsql upsert example. Per Phil Sorber, though I didn't use his wording exactly. http://git.postgresql.org/pg/commitdiff/ed6e0545f5f6e9977c8410e04244138b567c5a73
  • Fix pushing of index-expression qualifications through UNION ALL. In commit 57664ed25e5dea117158a2e663c29e60b3546e1c, I made the planner wrap non-simple-variable outputs of appendrel children (IOW, child SELECTs of UNION ALL subqueries) inside PlaceHolderVars, in order to solve some issues with EquivalenceClass processing. However, this means that any upper-level WHERE clauses mentioning such outputs will now contain PlaceHolderVars after they're pushed down into the appendrel child, and that prevents indxpath.c from recognizing that they could be matched to index expressions. To fix, add explicit stripping of PlaceHolderVars from index operands, same as we have long done for RelabelType nodes. Add a regression test covering both this and the plain-UNION case (which is a totally different code path, but should also be able to do it). Per bug #6416 from Matteo Beccati. Back-patch to 9.1, same as the previous change. http://git.postgresql.org/pg/commitdiff/b28ffd0fcc583c1811e5295279e7d4366c3cae6c
  • Tweak index costing for problems with partial indexes. btcostestimate() makes an estimate of the number of index tuples that will be visited based on knowledge of which index clauses can actually bound the scan within nbtree. However, it forgot to account for partial indexes in this calculation, with the result that the cost of the index scan could be significantly overestimated for a partial index. Fix that by merging the predicate with the abbreviated indexclause list, in the same way as we do with the full list to estimate how many heap tuples will be visited. Also, slightly increase the "fudge factor" that's meant to give preference to smaller indexes over larger ones. While this is applied to all indexes, it's most important for partial indexes since it can be the only factor that makes a partial index look cheaper than a similar full index. Experimentation shows that the existing value is so small as to easily get swamped by noise such as page-boundary-roundoff behavior. I'm tempted to kick it up more than this, but will refrain for now. Per report from Ruben Blanco. These are long-standing issues, but given the lack of prior complaints I'm not going to risk changing planner behavior in back branches by back-patching. http://git.postgresql.org/pg/commitdiff/21a39de5809cd3050a37d2554323cc1d0cbeed9d
  • Fix typo in comment. Peter Geoghegan http://git.postgresql.org/pg/commitdiff/dd243b3e40c24cd7c6b0db80bb39061f8f85af7b
  • Assorted comment fixes, mostly just typos, but some obsolete statements. YAMAMOTO Takashi http://git.postgresql.org/pg/commitdiff/ad10853b30b84d89905e023afa599de3a1fea4c6

Peter Eisentraut a poussé :

  • Remove quotes around format_type_be() output. format_type_be() takes care of any needed quoting itself. http://git.postgresql.org/pg/commitdiff/89dda5f2979fbe277809369ff88832ab39e83ff0
  • Do not access indclass through Form_pg_index. Normally, accessing variable-length members of catalog structures past the first one doesn't work at all. Here, it happened to work because indnatts was checked to be 1, and so the defined FormData_pg_index layout, using int2vector[1] and oidvector[1] for variable-length arrays, happened to match the actual memory layout. But it's a very fragile assumption, and it's not in a performance-critical path, so code it properly using heap_getattr() instead. bug analysis by Tom Lane http://git.postgresql.org/pg/commitdiff/8a3f745f160d8334ad978676828d3926ac949f43
  • Hide most variable-length fields from Form_pg_* structs. Those fields only appear in the structs so that genbki.pl can create the BKI bootstrap files for the catalogs. But they are not actually usable from C. So hiding them can prevent coding mistakes, saves stack space, and can help the compiler. In certain catalogs, the first variable-length field has been kept visible after manual inspection. These exceptions are noted in C comments. reviewed by Tom Lane http://git.postgresql.org/pg/commitdiff/8137f2c32322c624e0431fac1621e8e9315202f9
  • Disallow ALTER DOMAIN on non-domain type everywhere. This has been the behavior already in most cases, but through omission, ALTER DOMAIN / OWNER TO and ALTER DOMAIN / SET SCHEMA would silently work on non-domain types as well. http://git.postgresql.org/pg/commitdiff/27874583627e049a049dc1327deb12a02a7013ab
  • Revert unfortunate whitespace change. In e5e2fc842c418432756d8b5825ff107c6c5fc4c3, blank lines were removed after a comment block, which now looks as though the comment refers to the immediately following code, but it actually refers to the preceding code. So put the blank lines back. http://git.postgresql.org/pg/commitdiff/bf90562aa464e3a9afedde5f0007058f381d00fe
  • Show default privileges in information schema. Hitherto, the information schema only showed explicitly granted privileges that were visible in the *acl catalog columns. If no privileges had been granted, the implicit privileges were not shown. To fix that, add an SQL-accessible version of the acldefault() function, and use that inside the aclexplode() calls to substitute the catalog-specific default privilege set for null values. reviewed by Abhijit Menon-Sen http://git.postgresql.org/pg/commitdiff/b376ec6fa57bc76037014ede29498e2d1611968e

Alvaro Herrera a poussé :

  • Add pg_trigger_depth() function. This reports the depth level of triggers currently in execution, or zero if not called from inside a trigger. No catversion bump in this patch, but you have to initdb if you want access to the new function. Author: Kevin Grittner http://git.postgresql.org/pg/commitdiff/74ab96a45ef6259aa6a86a781580edea8488511a
  • Have \copy go through SendQuery. This enables a bunch of features, notably ON_ERROR_ROLLBACK. It also makes COPY failure (either in the server or psql) as a whole behave more sanely in psql. Additionally, having more commands in the same command line as COPY works better (though since psql splits lines at semicolons, this doesn't matter much unless you're using -c). Also tighten a couple of switches on PQresultStatus() to add PGRES_COPY_BOTH support and stop assuming that unknown statuses received are errors; have those print diagnostics where warranted. Author: Noah Misch http://git.postgresql.org/pg/commitdiff/08146775acd8bfe0fcc509c71857abb928697171

Magnus Hagander a poussé :

Heikki Linnakangas a poussé :

Correctifs rejetés (à ce jour)

  • Pas de déception cette semaine :-)

Correctifs en attente

  • Noah Misch sent in another revision of the patch to collect statistics for arrays.
  • Jim Mlodgensky and Heikki Linnakangas traded patches to add a new GUC for welcome messages.
  • Matthew Draper sent in another revision of the patch to enable SQL language functions to reference parameters by name.
  • Simon Riggs sent in another revision of the patch to have a WAL restore process during recovery.
  • Simon Riggs sent in another revision of the freelist_wait_stats patch.
  • Alexander Korotkov and Jeff Davis traded patches for GiST indexing on range types.
  • Alvaro Herrera sent in two more revisions of the FOREIGN KEY LOCK patch.
  • Jaime Casanova sent in another revision of the relation_free_space patch.
  • Thomas Ogrisegg sent in a patch intended to fix a bug in pg_basebackup, which exits with 0 even if it had errors while writing the backup to disk when the backup file is to be sent to stdout.
  • Fujii Masao sent an add-on patch to the one which adds a "write" synchrounous replication mode to modify postgresql.conf appropriately.
  • Noah Misch sent in another revision of the patch to prevent unneeded table rewrites in certain types of ALTER TABLE...ALTER TYPE operations.
  • Heikki Linnakangas sent in two more revisions of the GROUP COMMIT patch, reviews by Robert Haas, Jeff Janes and Jesper Krogh.
  • Dimitri Fontaine sent in another revision of the patch to add triggers to commands in general.
  • Abhijit Menon-Sen and Pavel Stehule traded revisions of a patch to allow better debugging of overloaded functions.
  • Abhijit Menon-Sen sent in a patch atop Andrew Dunstan's patch to add {query,array,row}-to-json functionality.
  • KaiGai Kohei sent in another revision of the patch to plug certain types of information leaks in views.
  • Abhijit Menon-Sen sent in a patch to fix some infelicities in Peter Eisentraut's patch to enable using \0 as a field or record separator.
  • Jaime Casanova sent in another revision of the patch to add a pg_stats_recovery view.
  • Abhijit Menon-Sen sent in a fix to Daniel Farina's patch which factors out the various crc32 implementations in the code base.
  • Kyotaro HORIGUCHI sent in another revision of the patch to add a new libpq tuple storage and use same to speed up dblink. Reviews by Marko Kreen and Merlin Moncure.
  • Peter Geoghegan sent in another revision of the fast path sort patch.
  • Marti Raudsepp sent in another revision of the patch to cache stable expressions with constant arguments.
  • Luben Karavelov sent in a patch to enable limited cover density ranking in text search. Reviews by Oleg Bartunov and Sushant Sinha.
  • Dan Scales sent in another WIP patch implementing double-write with the suggested double-write buffers per feedback from Heikki Linnakangas and Simon Riggs.
  • Jeff Janes sent in a patch to allow simulating log contention in pg_bench.
  • Dean Rasheed sent in a patch to fix a performance regression in index-only scans.
  • Petr (PJMODOS) Jelinek sent in another revision of the patch to enable CHECK FUNCTION.
  • Simon Riggs sent in another revision of the DROP INDEX CONCURRENTLY patch.
  • Simon Riggs sent in another revision of the patch intended to reduce CLOG contention.

par N Bougain le jeudi 2 février 2012 à 01h46

lundi 30 janvier 2012

Nicolas Thauvin

Got GUC?

Les paramètres de configuration de PostgreSQL sont appelés GUC ce qui signifie Grand Unified Configuration, c'est le nom de la partie du code qui gère les paramètres de configuration. En gros, ce sont tous les paramètres du fichier postgresql.conf.

Ce qui est moins connu et utilisé, c'est la possibilité de configurer ces paramètres à différents niveaux :

  1. Fichier postgresql.conf
  2. Ligne de commande du postmaster, le processus principal du serveur
  3. Base de données
  4. Rôle
  5. Session
  6. Transaction

La précédence des valeurs va en descendant dans la liste, par exemple la valeur d'un paramètre au niveau d'un rôle écrase celle positionnée au niveau de la base de donnée ou la ligne de commande. Ce comportement est très intéressant pour définir une valeur d'un paramètre dépendante du contexte d'exécution d'un traitement. Par exemple on peut placer un timeout des requêtes au niveau de la base pour éviter qu'une application ne jette l'éponge avant PostgreSQL, et configurer l'absence de timeout pour un rôle dédié aux opérations de VACUUM et ANALYSE, on limite ainsi l'effet de bord du timeout :

-- timeout à 30 secondes sur la base de données
ALTER DATABASE mabase SET statement_timeout TO 30000;

-- pas de timeout pour le role maintenance chargé du vacuum
ALTER ROLE maintenance SET statement_timeout TO 0;

Selon l'endroit où doit être positionné la valeur on utilise :

  • postgresql.conf : directement dans le fichier
  • ligne de commande : dans le script d'init avec l'option -c et à l'exécution de pg_ctl avec l'option -o
  • base de données : ALTER DATABASE nom_base SET param TO valeur;
  • rôle : ALTER ROLE nom_role SET param TO valeur;
  • session : SET [ SESSION ] param TO valeur;
  • transaction : SET LOCAL param TO valeur;

Pour le passage des valeurs au niveau SQL, on peut utiliser ... RESET param à la place de SET param TO pour réinitialiser la valeur à son défaut pour le contexte choisi.

On peut également définir des paramètres personnalisés, comme le font certaines extensions. Pour cela il faut définir une classe de variables personnalisée, en déclarant un préfixe (on en sépare plusieurs par des virgules) dans le paramètre de configuration custom_variable_classes :

custom_variable_classes = 'nico'

Ensuite, on peut directement ajouter nos variables personnalisées en les préfixant par nico. :

nico.test_guc = 1000

On peut alors manipuler ces variables comme ceci :

mydb=# SHOW nico.test_guc;
 nico.test_guc 
---------------
 1000
(1 row)

mydb=# SHOW nico.test_guc;
 nico.test_guc 
---------------
 1000
(1 row)

mydb=# SET nico.test_guc = 3;
SET
mydb=# SHOW nico.test_guc;
 nico.test_guc 
---------------
 3
(1 row)

mydb=# SET nico.reguc = on;
SET
mydb=# SHOW nico.reguc;
 nico.reguc 
------------
 on
(1 row)

Enfin, on peut utiliser les fonctions current_setting() et set_config() pour manipuler ces variables dans des fonctions :

mydb=# SELECT set_config('nico.test_guc', '100', false);
 set_config 
------------
 100
(1 row)

mydb=# SELECT current_setting('nico.test_guc');
 current_setting 
-----------------
 100
(1 row)

PS: merci à ce post pour l'idée de creuser le sujet.

par Orgrim le lundi 30 janvier 2012 à 16h41

mercredi 25 janvier 2012

Damien Clochard

Session PostgreSQL #3 : il ne reste plus qu'une poignée de places !

Visiblement nous avons visé juste ! En choisissant la Réplication comme thème de la prochaine Session PostgreSQL nous savions que le public répondrait nombreux mais nous n'avions pas prévu un tel engouement... Le conférence qui se tiendra le 2 février à Paris affiche quasiment complet ! Au moment ou j'écris ces lignes , il ne reste plus qu'une poignée de places disponibles. Si vous êtes intéressé par les problématiques de Haute-Disponibité sous PostgreSQL, je vous encourage à vous inscrire le plus rapidement possible à cette adresse :

http://www.postgresql-sessions.org/3/registration_form

Pour rappel, la session se tiendra de 9h30 à 17h30 au Comptoir Général situé 80 quai de Jemmapes à Paris ( http://osm.org/go/0BPIqc7Q )

Parmi les orateurs, nous aurons la chance de compter :

  • Mickaël Paquier (Japon), Développeur de Postgres-XC
  • Simon Riggs (Angleterre), Contibuteur majeur de PostgreSQL
  • Ludovic Levesque (France), Directeur Technique de Fotolia

Le programme complet est disponible ici : http://www.postgresql-sessions.org/3/

Rendez-vous le 2 février à Paris !

par damien le mercredi 25 janvier 2012 à 19h26

lundi 23 janvier 2012

Damien Clochard

FOSDEM 2012 : Demandez le programme !

Le FOSDEM 2012 se tiendra du 4 au 5 février à Bruxelles. C'est un rendez-vous important de la communauté européenne de PostgreSQL et comme d'habitude vous retrouverez pas mal de francophones derrière le stand de l'association PostgreSQL Europe !

Rappelons que le FOSDEM est un événement gratuit et sans inscriptions. Plus d'informations par ici : http://fosdem.org/2012/

Cette année encore, il y a une journée entière de conférences (en anglais) dédiée à PostgreSQL :

http://fosdem.org/2012/schedule/track/postgresql_devroom

Rendez-vous à Bruxelles le 4-5 février !

par damien le lundi 23 janvier 2012 à 20h16

dimanche 22 janvier 2012

Stephane Bortzmeyer

Changer une base PostgreSQL de « tablespace »

Un des principaux mécanismes de gestion de l'espace disque dans PostgreSQL est le tablespace (http://www.postgresql.org/docs/8.4/interactive/manage-ag-tablespaces.html). Un tablespace est un répertoire où on place des données du SGBD. Mais, si on change d'avis, comment changer une base de tablespace ?

dimanche 22 janvier 2012 à 00h00

jeudi 12 janvier 2012

Ismaila Baradji

Execution de requetes sur une base de données distante avec postgresql

L’acces a une base de donnees distante sur le SGBD PostgreSQL est distincte de celle de mysql. Pour le faire, plusieurs etapes sont necessaires: 1- Executer le script dblink.sql dans le repertoire share\contrib\ de postgres. ce qui va generer plusieurs fonctions permetant d’utiliser dblink 2- Connexion a la base SELECT * from dblink_connect(‘c1′,’hostaddr=192.168.7.8 port=5432 dbname=db_mydb …

Lire la Suite »

par Baradji le jeudi 12 janvier 2012 à 20h07

mercredi 4 janvier 2012

Damien Clochard

Session PostgreSQL #3 le 2 février: Réplication PostgreSQL

Dalibo organise le jeudi 2 février 2012 à Paris une rencontre internationale consacrée au Système de Gestion de Bases de Données PostgreSQL.

Le thème de cette journée sera la réplication, avec des invités de marques notamment :

 * Mickaël Paquier (Japon), Développeur de Postgres-XC
 * Simon Riggs (Angleterre), Contibuteur majeur de PostgreSQL
 * Ludovic Levesque (France), Directeur Technique de Fotolia

L'objectif de cette conférence est de faire un tour d'horizon des solutions de Haute-disponibilités : PGXC, Slony, Londiste et avec bien sûr un accent particulier la réplication interne avec le Hot Standby / Streaming Replication !

Retrouvez le programme complet sur le site de l'évènement : http://www.postgresql-sessions.org/...

La session se tiendra de 9h30 à 17h30 au Comptoir Général situé 80 quai de Jemmapes à Paris ( http://osm.org/go/0BPIqc7Q )

Cet événement est gratuit et ouvert à tous, dans la limite des places disponibles.

Les inscriptions se font via la page ci-dessous : http://www.postgresql-sessions.org/...

Pour toute précision, n'hésitez pas à envoyer un message à contact@postgresql-sessions.org

Bonne journée et Rendez-vous le 2 février juin à Paris !


À propos des Sessions PostgreSQL : Les sessions PostgreSQL sont avant tout des moments pour découvrir et rencontrer la communauté PostgreSQL. Chaque session est une journée composée de conférences et d'ateliers, organisée autour d'un thème précis et d'un invité de marque.

» site web : http://www.postgresql-sessions.org/


À propos de Dalibo : Spécialiste français de PostgreSQL, Dalibo met à la disposition de ses clients son savoir-faire dans le domaine des bases de données et propose des services de conseil, de formation et de support aux entreprises et aux institutionnels.

» site web : http://www.dalibo.com/

par damien le mercredi 4 janvier 2012 à 14h18

lundi 12 décembre 2011

Dimitri Fontaine (2nd Quadrant)

Pourquoi choisir PostgreSQL plutôt que SQL Server?

Un article récent en anglais propose dix bonnes raisons de choisir PostgreSQL plutôt que SQL Server dans le cadre d’un nouveau projet, ce qui nous offre une belle occasion de revenir sur notre publication précédente qui ciblait la migration vers PostgreSQL. Nous pouvons donc cette fois nous intéresser de plus prêt aux particularités de PostgreSQL !

La dizaine de points choisis par Jeremiah Peschka dans l’article précédent s’éloigne parfois de la liste que j’aurais moi-même établie, aussi vais-je éviter de vous faire une traduction simple du contenu anglais mentionné. Plusieurs points méritent tout de même d’être mentionnés ici.

PostgreSQL stabilise et distribue une nouvelle version majeure de sa base de données chaque année. Le cycle de développement actuel (que nous connaissons bien ici, pour y participer) est composé de deux grandes étapes, six mois de développement suivis de six mois de relectures, tests, correctifs, polissages et améliorations de la documentation.

Cela permet d’obtenir de très bonnes versions point zéro autour du mois de septembre chaque année. Cela n’a pas toujours été le cas dans la gestion du projet PostgreSQL, mais ce modèle a été mis en place de manière professionnelle sur plusieurs années afin de pouvoir mieux accepter les améliorations apportées par plus de deux cents développeurs en moyenne.

Bien sûr, très peu de projets industriels peuvent se permettre de revoir leur architecture, leurs procédures et leur intégration SQL chaque année, et même dans le milieu très dynamique des services web cela n’arrive quasiment pas.Aussi les versions de PostgreSQL sont-elles maintenues pendant au moins cinq ans, les versions courantes de PostgreSQL sont donc au nombre de 5 à 7 selon les moments de l’année.

L’avantage n’est donc pas de pouvoir mettre à jour chaque année, mais bien de pouvoir en toute sérénité choisir la date de mise à jour de la version de PostgreSQL selon son propre calendrier de maintenance et d’évolutions avec la garantie de pouvoir à tout moment choisir une nouvelle version vieille au maximum d’un an.

 

Il existe ensuite de nombreux points techniques donnant un avantage très net à PostgreSQL, soit qu’il s’agisse d’innovations technologiques issues de la recherche, telles les « Serializable snapshot isolation » (ou SSI), ou bien la réplication synchrone contrôlable pour chaque transaction ; ou bien qu’il s’agisse de supports avancés aux développeurs, telles les recherches des plus proches voisins via un parcours d’index, les requêtes avancées en écriture (comparable au pipe sous unix, mais avec des ordres de lecture ou d’écriture SQL), ou bien le support avancé des types de données souvent utilisés.

Ces capacités avancées, cette souplesse d’utilisation et de paramétrage dynamique, associés à des caractéristiques de performance époustouflantes (dans la plupart des usages, dont très certainement le vôtre), la qualité de sa documentation, tout cela donne un avantage crucial à PostgreSQL : une réduction imbattable des coûts de développement et de maintenance de vos logiciels.

Vos développeurs peuvent écrire des requêtes complexes avec la garantie d’obtenir le bon résultat même dans une utilisation concurrente, faire des calculs de dates dans des timezones différentes et au milieu du changement d’heure (tous les mois ne font pas le même nombre de jours, tous les jours ne font pas 24 heures non plus, PostgreSQL le sait), exprimer des contraintes avancées (le même client ne doit pas réserver deux voitures différentes dans le même créneau horaire) qui doivent fonctionner systématiquement et sans verrous, etc.

Bénéficier de PostgreSQL pour résoudre cet ensemble de problème permet de ne pas avoir à les résoudre à nouveau dans votre application (quel jour serons-nous dans trois mois ? SELECT to_char(date 'today' + 3 * interval '1 month', 'Day'); est sûrement plus facile à utiliser que n’importe quel autre code, les développeurs lisant cela seront sûrement d’accord). Le temps passé à exploiter la documentation de PostgreSQL est un investissement facile à réutiliser et à rentabiliser, et c’est du temps de gagné dans le développement de ce qui fait la valeur ajoutée de votre application.

La raison pour laquelle nous aimons tant vanter les qualités techniques de PostgreSQL est simple. Il s’agit là d’un formidable levier vous permettant de produire des application meilleures car plus simple à développer, à faire évoluer, à maintenir, déployer et administrer.

Bien évidemment, tout cela sans s’acquitter de coûts de licence appliqués par serveur ou qui dépendent de la capacité et du nombre d’installations dont vous avez besoin pour déployer une architecture. Mettre en place une solution tolérante aux pannes, de la répartition de charge pour vos rapports hebdomadaires, des instances de tests ou d’exports de données devient une décision plus facile à prendre car elle ne dépend pas des coûts de licence.

Il reste à former une équipe compétente, ce qui à mon sens est une bien meilleure utilisation de votre budget, et ce qui bien souvent représente un coût moins élevé que les licences dépensées en pure perte… dès lors que vous pouvez obtenir le même service à moindre coût. Nous pouvons vous aider à déterminer si PostgreSQL est le moteur de bases de données le mieux adapté à votre projet.

par Dimitri Fontaine le lundi 12 décembre 2011 à 13h58

Christophe Chauvet

Connaitre la taille d'un base de données PostgreSQL

Pour connaitre la taille d'un base de données il faut utiliser la fonction pg_database_size

production=# select pg_database_size('production');
 pg_database_size
------------------
        513343780
(1 ligne)

Cette taille est donnée en octets, pour avoir une meilleur représentation en Méga ou Giga, il faut utiliser la fonction pg_size_pretty

production=# select pg_size_pretty(pg_database_size('production'));
 pg_size_pretty
----------------
 490 MB
(1 ligne)

Ensuite si l'on souhaite connaître la taille d'un table il faut utiliser la fonction pg_relation_size.

production=# select pg_size_pretty(pg_relation_size('res_partner'));
 pg_size_pretty
----------------
 152 kB
(1 ligne)

Si l'on souhaite également avoir la place prise par les indexes, il faut utiliser la fonction pg_total_relation_size

production=# select pg_size_pretty(pg_total_relation_size('res_partner'));
 pg_size_pretty
----------------
 528 kB
(1 ligne)

par Christophe Chauvet le lundi 12 décembre 2011 à 11h30

samedi 10 décembre 2011

Guillaume Lelarge

Nouvelles versions mineures, mise à jour de la traduction française

Contrairement à mon habitude, les manuels français de PostgreSQL n'ont été mis à jour que maintenant, soit cinq jours après la sortie des versions. Pour me faire pardonner, j'ai enfin corrigé le problème de la recherche dans la documentation de la version 9.1 (merci à Thomas pour l'info).

J'allais oublier... la documentation de la 8.2 n'a pas disparu. Elle est juste partie rejoindre les documentations des versions obsolètes.

par Guillaume Lelarge le samedi 10 décembre 2011 à 16h01

lundi 7 novembre 2011

Base-sql.fr

Install postgresql avec yum et modif de port !

Bonjour,

J’ ai constaté une petite coquille avec l’installation de postgresql 9.0.x avec Yum sur Centos (par exemple).
On a beau changer de port au niveau du postgresql.conf ça ne change en aucun cas le port d’écoute :)
Il faut aller dans le /etc/init.d/postresql-9.0 et modifier le PGPORT à la main.

Et il ne faut pas oublier qu’en cas de mise à jour le port par défaut (5432) reviendra dans le fichier de démarrage, donc à remodifier.

Voilà je voulais juste le signaler ^^

edit suite a un commentaire fort intéressant que je recopie ici pour plus de visibilité :

j’en profite pour noter une petite astuce à base de variables d’environnement, avec les paquets PG ( http://yum.postgresql.org/ )

Il suffit de linker le script dans /etc/init.d, et de créer le fichier de config correspondant dans /etc/sysconfig/pgsql/

Article détaillé sur le blog de 2nd Quadrant :
http://blog.2ndquadrant.com/en/2010/05/install-multiple-postgresql-servers-redhat-linux.html
Et quelques infos sur le wiki :
http://wiki.postgresql.org/wiki/PostgreSQL_on_RedHat_Linux

par kenrio le lundi 7 novembre 2011 à 16h26

lundi 31 octobre 2011

Dimitri Fontaine

Extensions en simple SQL

La conférence européenne à Amsterdam était un très bon évènement de la communauté, avec une organisation impeccable dans un hôtel accueillant. J'ai eu le plaisir d'y parler des extensions et de leur usage dans le cadre du développement applicatif « interne », sous le titre Extensions are good for business logic.

L'idée de ma présentation, que la plupart d'entre vous a loupé je suppose (en tout cas je n'avais qu'une petite poignée de français dans la salle, et j'espère avoir des lecteurs qui n'étaient pas à Amsterdam), l'idée est d'utiliser les mécanismes offerts par les extensions afin de maintenir le code PL que vous utilisez en production.

Il s'agit la plupart du temps de procédures qui implémentent une partie de la logique métier de vos applications, mais si proche des données que cela termine en base directement : c'est une bonne chose, en particulier depuis PostgreSQL 9.1. Cette version propose en effet une gestion assez complète des extensions.

Il s'agit de réaliser un empaquetage de vos procédures en suivant la documentation en ligne et son chapitre 35.15. Empaqueter des objets dans une extension. Une fois cela fait, il est alors possible de déployer votre ensemble de procédure stockée avec la commande CREATE EXTENSION mesprocs;, et ensuite la commande psql \dx vous permet de lister les extensions installées et leur numéro de version.

Les mises à jours sont également gérées avec une commande SQL dédiée, il s'agit alors de ALTER EXTENSION mesprocs UPDATE [TO version];. Il suffit de fournir des scripts intermédiaires nommés par exemple mesprocs--1.0--1.1.sql et mesprocs--1.1--1.2.sql et PostgreSQL saura comment passer de 1.0 à 1.1.

Voilà, vous savez presque tout de ma présentation à Amsterdam et vous pouvez retrouver le reste sur le support proposé en début d'article. Bien sûr je n'ai pas reproduit ici les questions intéressantes qui m'ont été posées, une bonne partie d'entre elles sont venues enrichir ma liste de Noël pour les extensions. Si vous voulez être sûr de trouver cela sous votre sapin, cependant, le meilleur moyen est encore de m'en parler : sponsoriser les développement Open Source est une belle démarche :)

par dim@tapoueh.org (Dimitri Fontaine) le lundi 31 octobre 2011 à 13h22