How To Enable Variable References in Perl While Using Strict

I wanted to call a sub-routine based on a variable in Perl, like this:

but it failed because of use strict:

To enable using a variable as a reference, simply specify that to Perl:

Categories: How-To's, Technology Tags: de-reference, dereference, howto, no strict, perl, Reference, Strict, Subroutine, tips, Variable | No comments

How To Print Matching Line and All After Using awk

Every time I use awk, my respect for it grows! Like grep, find and rsync, it is one of the powerhouse command-line tools in my arsenal.

Recently, I needed to list all MySQL binary log files from a starting file forwards in sequence. AWK to the rescue!

For example, I knew the starting file was mysql-bin.000325, and also that there is a list of binary log files kept by MySQL called mysql-bin.index, so I executed the following to extract the lines I wanted out of the 76 available in the file:

Per the man page:
“Two patterns separated by a comma is called a range pattern. With it, once the first pattern is matched, every subsequent record matches until the second pattern is matched.”

We can call the first argument the “starting match pattern” and the second argument the “ending match pattern“.

Awk will start to print lines (the default behavior for a match) when it finds the starting match pattern and end after it finds and prints the ending match pattern line.

Since the ending match pattern is zero, it will never be true and so awk simply prints all remaining lines in the source.

Categories: How-To's, Technology Tags: AWK, binary log file, howto, Index, Log, log file, Match, mysql, tips | No comments

Tungsten Clustering 5.4.0 and Tungsten Replicator 5.4.0 Released

Continuent is pleased to announce that the following new software releases are now available:

• Tungsten Clustering version 5.4.0
• Tungsten Replicator version 5.4.0

Releases 5.4.0 is significant in that it introduces MySQL 8 support, along with many new features, stability improvements and bug fixes.

Highlights common to both products:

Improvements, new features and functionality

• Two new utility scripts have been added to the release to help with setting the Replicator position:
  • tungsten_find_position, which assists with locating information in the THL based on the provided MySQL binary log event position and outputs a dsctl set command as output.
  • tungsten_find_seqno, which assists with locating information in the THL based on the provided sequence number and outputs a dsctl set command as output.
• A new, beta-quality command has been included called prov-sl.sh which is intended to eventually replace the current tungsten_provision_slave script.
  Currently, prov-sl.sh supports provisioning slaves using mysqldump and xtrabackup tools, and is MySQL 8-compatible.  
The prov-sl.sh command is written in Bash, has less dependencies compared to the current script and is meant to fix a number of issues with the current version. 
Backups are streamed from source to target so that an intermediate write to disk is not performed, resulting in faster provisioning times.
• Upgraded the Drizzle driver to support MySQL 8 authentication protocols (SHA256, caching_sha2)
• The replicator has been updated to support the new character sets supported by MySQL 5.7 and MySQL 8.0, including the UTF-8-mb4 series.

Bug Fixes and Behavior Changes

• The tpm command now properly handles network interface names containing colons and/or dots.
• The tpm diag --hosts={hostlist} command has been fixed. In a Staging-method install, using this option limits the tpm diag command to the specified hosts. In an INI-based install, tpm diag by itself gets diags for the local node only, and specifying the option --hosts={hostlist} will attempt to obtain diags from the named hosts only.
• When using tpm with the INI method, the command would search multiple locations for suitable INI files. This could lead to multiple definitions of the same service, which could in turn lead to duplication of the installation process and occasional failures. If multiple INI files are found, a warning is now produced to highlight the potential for failures.
• When executing mysqldump, all Tungsten tools no longer use the –add-drop-database flag as it will prevent MySQL 8+ from restoring the dump.
• Now properly extracting the Geometry datatype.
• Added support for missing charset GB18030.

Highlights in the Clustering product:

Behavior Changes
The following changes have been made to Continuent Tungsten and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

• The Connector passThroughMode configuration option is now deprecated, and will be removed from tungsten-connector/conf/connector.properties. There is currently no tpm option for this, and it is undocumented. The default will be kept to passThroughMode=true.

Improvements, new features and functionality

• A new utility script has been added to the release, tungsten_post_process, which assists with the graceful maintenance of the static cross-site replicator configuration files on disk.
• A new utility script has been added to the release, tungsten_reset_manager</strong >, which assists with the graceful reset of the manager’s dynamic state files on disk.
• The Tungsten Stack now supports the new MySQL 8.0 authentication plugins. Both sha256_password and caching_sha2_password (the new default) are supported by the Replicator, Manager and Connector.
• The Drizzle driver has been updated to support these new authentication methods, and the MySQL Connector/J 8 is also supported.
• In order to be fully transparent with the new defaults, when connected to a MySQL 8+ data source, the Connector will advertise caching_sha2_password as the default plugin.
  With earlier versions of MySQL (pre-8.0), the previous default mysql_native_password is used by default and advertised to the client applications.
• There is a new Connector property option for tpm: property=statement.display.size.in.kb=NNN, which allows for increased verbosity when Connector logging is set to debug or trace.
• Removed spurious warnings during composite switch or failover.

Bug Fixes

• The Connector will now wait indefinitely for a Master to become available before finishing startup.
• Fixing the rpm-based post-install chown command so that symlinked directories get correct ownership.
• The Tungsten Clustering RPM now preserves the original OS group memberships for the tungsten user.
• Long service names within cctrl could cause output to fail when displaying information. The underlying issue has been fixed. Because long service names can cause formatting issues, a new option, --cctrl-column-width has been added which can be used to configure the minimum column width used to display information.
• MySQL ping commands are now reconnected/retried upon “server gone away” error (Proxy mode ONLY).
• Fixed an edge case where the master node and the coordinator node are the same, then the node was rebooted. The failover would not complete and throw an error.

Highlights for the Replicator product only:

• The trepctl command now properly handles the -all-services option for the reset sub-command.
• The Redshift Applier now allows AWS authentication using IAM Roles. Previously authentication was possible via Access and Secret Key pairs only.
• Loading data into Redshift would fail with an error if a row of data contained a specific control character (0x00 (NULL))
• The ddl_map.json file used by the apply_schema_changes filter was missing a rule to handle ALTER TABLE statements when replicating between MySQL and Redshift
• The extract_schema_change filter wasn’t escaping ” (double-quotes) and the generated JSON would then cause the applier to error out.

Release notes:

https://docs.continuent.com/tungsten-clustering-5.4/release-notes-5-4-0.html

https://docs.continuent.com/tungsten-replicator-5.4/release-notes-5-4-0.html

Categories: Continuent Tungsten, MySQL 8, News, Tungsten Clustering, Tungsten Replicator Tags: Clustering, mysql, release, Replicator | Comments Off on Tungsten Clustering 5.4.0 and Tungsten Replicator 5.4.0 Released

Tungsten Clustering 6.1.0 and Tungsten Replicator 6.1.0 Released

Continuent is pleased to announce that the following new software releases are now available:

• Tungsten Clustering version 6.1.0
• Tungsten Replicator version 6.1.0

Release 6.1.0 is significant in that it introduces MySQL 8 support, along with many new features, stability improvements and bug fixes.

Highlights common to both products:

Improvements, new features and functionality

• Two new utility scripts have been added to the release to help with setting the Replicator position:
  • tungsten_find_position, which assists with locating information in the THL based on the provided MySQL binary log event position and outputs a dsctl set command as output.
  • tungsten_find_seqno, which assists with locating information in the THL based on the provided sequence number and outputs a dsctl set command as output.
• A new, beta-quality command has been included called prov-sl.sh which is intended to eventually replace the current tungsten_provision_slave script.
  Currently, prov-sl.sh supports provisioning slaves using mysqldump and xtrabackup tools, and is MySQL 8-compatible.  
The prov-sl.sh command is written in Bash, has less dependencies compared to the current script and is meant to fix a number of issues with the current version. 
Backups are streamed from source to target so that an intermediate write to disk is not performed, resulting in faster provisioning times.
• Upgraded the Drizzle driver to support MySQL 8 authentication protocols (SHA256, caching_sha2)

Bug Fixes and Behavior Changes

• When executing mysqldump, all Tungsten tools no longer use the –add-drop-database flag as it will prevent MySQL 8+ from restoring the dump.
• Now properly extracting the Geometry datatype.
• Added support for missing charset GB18030.

Highlights for the Clustering product:

Behavior Changes
The following changes have been made to Continuent Tungsten and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

• The Connector passThroughMode configuration option is now deprecated, and will be removed from tungsten-connector/conf/connector.properties. There is currently no tpm option for this, and it is undocumented. The default will be kept to passThroughMode=true.

Improvements, new features and functionality

• A new utility script has been added to the release, tungsten_post_process, which assists with the graceful maintenance of the static cross-site replicator configuration files on disk.
• There is a new Connector property option for tpm: property=statement.display.size.in.kb=NNN, which allows for increased verbosity when Connector logging is set to debug or trace.
• The Tungsten Stack now supports the new MySQL 8.0 authentication plugins. Both sha256_password and caching_sha2_password (the new default) are supported by the Replicator, Manager and Connector.
• The Drizzle driver has been updated to support these new authentication methods, and the MySQL Connector/J 8 is also supported.
• In order to be fully transparent with the new defaults, when connected to a MySQL 8+ data source, the Connector will advertise caching_sha2_password as the default plugin.
  With earlier versions of MySQL (pre-8.0), the previous default mysql_native_password is used and advertised to the client applications.

Bug Fixes

• The Connector will now wait indefinitely for a Master to become available before finishing startup.

Highlights for the Replicator product only:

• The Redshift Applier now allows AWS authentication using IAM Roles. Previously authentication was possible via Access and Secret Key pairs only.
• Loading data into Redshift would fail with an error if a row of data contained a specific control character (0x00 (NULL))
• The ddl_map.json file used by the apply_schema_changes filter was missing a rule to handle ALTER TABLE statements when replicating between MySQL and Redshift
• The extract_schema_change filter wasn’t escaping ” (double-quotes) and the generated JSON would then cause the applier to error out.

Release notes:

https://docs.continuent.com/tungsten-clustering-6.1/release-notes-6-1-0.html

https://docs.continuent.com/tungsten-replicator-6.1/release-notes-6-1-0.html

Categories: Continuent Tungsten, MySQL 8, News, Tungsten Clustering, Tungsten Replicator Tags: Clustering, mysql, release, Replicator | Comments Off on Tungsten Clustering 6.1.0 and Tungsten Replicator 6.1.0 Released

How To Add a Simple Checkbox in Confluence

While using Confluence, I wanted to learn how to add a simple task checkbox in free-flow text and lists without creating a “real” task.

The answer is very easy – just type the square bracket pair [] and the checkbox will appear!

Categories: How-To's, Technology Tags: CheckBox, Confluence, howto, Simple, Task, tips | No comments

How To Fix File Sharing Problems in MacOS 10.14 Mojave

When I upgraded to Mojave, my laptops were no longer able to connect to my desktop. Deeply frustrating.

Long story short, combo of factors:
– Mojave filesystems on disk now default to APFS containers
– APFS filesystems can ONLY be shared out via SMB!
– Hostname resolution in Mojave also seems broken

So, the fix for me was to do the following on the calling laptops:

Go into the Finder app and press ⌘+K to open up the Connect to Server dialog:

Type: smb:// followed by the server’s IP address:

You should be presented with a list of available volumes to choose from:

Once the sharing is working via the IP address, next fix and test via the hostname:

sudo vi /etc/hosts
Add entries for all hosts you wish to connect TO, then save and exit.

Go into the Finder app and press ⌘+K to open up the Connect to Server dialog, and enter smb:// followed by the server’s hostname:

This should also result in a list of volumes to be selected from.

Once that happens you are Good To Go!

As always, YMMV!

Categories: How-To's, Technology Tags: AFS, APFS, Default, File sharing, Finder, howto, MacOS, Mojave, Sharing, SMB, tips | No comments

How to move the Relay role to another node in a Composite Tungsten Cluster

The Question

Recently, a customer asked us:

How would we manually move the relay role from a failing node to a slave in a Composite Tungsten Cluster passive site?

---

The Answer

The Long and the Short of It

There are two ways to handle this procedure manually when the usual switch command fails to work as expected. One is short and reasonably automated, and the other is much more detailed and manual.

Of course, the usual procedure is to just issue the switch command in the passive cluster:

[crayon-5db0bf70aa36e261515292/]

The below article describes what to do when the switch command does not move the relay role to another node.

SHORT

Below is the list of cctrl commands that would be run for the basic, short version, which (aside from handling policy changes) is really only three key commands:

[crayon-5db0bf70aa374409312386/]

LONG

Below is the list of cctrl commands that would be run for the extended, manual version:

[crayon-5db0bf70aa376170049102/]

Full Procedure for SHORT (Automatic)

First, enable Maintenance mode to keep the Manager from interfering:

[crayon-5db0bf70aa378522745676/]

Tell the cluster that node db4 is failed using the datasource {node} fail command:

[crayon-5db0bf70aa37a041976613/]

Here is the state of all nodes after failing node db4:

[crayon-5db0bf70aa37c911003597/]

Next, tell the cluster to pick a new relay automatically using the failover command:

[crayon-5db0bf70aa37f071317726/]

Here is the state of all nodes after performing the failover:

[crayon-5db0bf70aa385335655858/]

Now “fix” the db4 node and welcome it back into the cluster using the recover command:

[crayon-5db0bf70aa389947886715/]

Here is the state of all nodes after performing the recover command:

[crayon-5db0bf70aa38c586631454/]

Finally, return the cluster to Automatic mode so the Manager will detect problems and react automatically.

[crayon-5db0bf70aa38f768508370/]

Full Procedure for LONG (Manual)

In the below example, node db6 is the current relay with db4 and db5 as slaves.

To force a current slave (i.e. db4) to become a relay to take over from db6 manually, follow the below example:

[crayon-5db0bf70aa391680584156/]

STEP 1. Shun node db6, the current relay

Tell the cluster to shun the current relay node db6 using the datasource {node} shun command:

[crayon-5db0bf70aa393443863000/]

Here is the state of all nodes after performing the shun command:

[crayon-5db0bf70aa395271409364/]

STEP 2. Process node db4, the new relay

Take datasource db4 offline and change the role to relay:

[crayon-5db0bf70aa39a417153383/]

Here is the state of all nodes after performing the offline and relay commands:

[crayon-5db0bf70aa39c344298647/]

STEP 3. Process node db5, a slave

Tell the replicator on db5 to go offline, then configure it to be a slave of new relay node db4, and then bring it right back online:

[crayon-5db0bf70aa39f749024337/]

Here is the state of all nodes after performing the commands:

[crayon-5db0bf70aa3a3286789277/]

STEP 4. Process node db6, the OLD relay and configure it to be a slave of db4, the NEW relay

Bring the replicator on node db6 offline:

[crayon-5db0bf70aa3a6558106217/]

Change the role of the replicator on node db6 to a slave of node db4 (the new relay):

[crayon-5db0bf70aa3aa995974821/]

Bring the replicator on node db6 online:

[crayon-5db0bf70aa3ac581278621/]

Change the role of datasource db6 to a slave:

[crayon-5db0bf70aa3ad645050950/]

Welcome datasource db6 back into the cluster. Once welcomed, it will be in the OFFLINE state.

[crayon-5db0bf70aa3af176781067/]

Bring datasource db6 online:

[crayon-5db0bf70aa3b1314986853/]

At this point the cluster is completely back to healthy and in the desired configuration:

[crayon-5db0bf70aa3b3261232529/]

Finally, return the cluster to Automatic mode so the Manager will detect problems and react automatically.

[crayon-5db0bf70aa3b6323171022/]

---

The Library

Please read the docs!

For more information about using various cctrl commands, please visit the docs page at https://docs.continuent.com/tungsten-clustering-6.0/cmdline-tools-cctrl-commands.html

For more information about Tungsten clusters, please visit https://docs.continuent.com.

---

Summary

The Wrap-Up

In this blog post we discussed two ways to move the relay role to another node in a Composite Tungsten Cluster.

Tungsten Clustering is the most flexible, performant global database layer available today – use it underlying your SaaS offering as a strong base upon which to grow your worldwide business!

For more information, please visit https://www.continuent.com/solutions

Want to learn more or run a POC? Contact us.

---

Categories: Architecture, cctrl, cls, Composite, Datasource, failover, Maintenance, Mastering Tungsten Clustering, Question Tags: Automatic, Command, HA, High Availability, Master, Move, mysql, QA, Relay, Replicator, Shell, Slave, Switch, Tungsten | Comments Off on How to move the Relay role to another node in a Composite Tungsten Cluster

How can I tell which Tungsten Connector mode I am using: Bridge, Proxy/Direct or Proxy/SmartScale?

Overview

The Skinny

Part of the power of Tungsten Clustering for MySQL / MariaDB is its intelligent MySQL Proxy, known as the Tungsten Connector. Tungsten Connector has three main modes, and depending on the type of operations you are performing (such as if you need read-write splitting), we help you choose which mode is best.

---

The Question

Recently, a customer asked us:

How can I tell which Tungsten Connector mode I am using: Bridge, Proxy/Direct or Proxy/SmartScale?

---

The Answer

Connect and Observe

You may login through the Connector to tell the difference between Bridge mode and Proxy mode (either Direct or SmartScale):

In Proxy mode, you will see the -tungsten tag appended to the Server version string:

Once logged into the Connector in Proxy mode, you have the full set of interactive tungsten commands available:

For more information about the Connector’s command-line interface, please visit http://docs.continuent.com/tungsten-clustering-6.0/connector-inline.html

For Bridge mode, you will not see that:

In Bridge mode, the tungsten commands do not work:

The Library

Please read the docs!

For more information about the Tungsten Connector:

• Blog Post: MySQL Client Connection Handling in the Tungsten Connector
• Documentation: Connector Routing Types

For more documentation about Tungsten software, please visit https://docs.continuent.com

---

Summary

The Wrap-Up

In this blog post we discussed how one can tell which Tungsten Connector mode is in use: Bridge, Proxy/Direct or Proxy/SmartScale.

Tungsten Clustering is the most flexible, performant global database layer available today – use it underlying your SaaS offering as a strong base upon which to grow your worldwide business!

For more information, please visit https://www.continuent.com/solutions

Want to learn more or run a POC? Contact us

Categories: Bridge, Cluster, Connector, Direct, Mastering Tungsten Clustering, SmartScale Tags: Clustering, mysql, proxy, Tungsten | Comments Off on How can I tell which Tungsten Connector mode I am using: Bridge, Proxy/Direct or Proxy/SmartScale?

Slick Command-Line Tricks for a Tungsten MySQL / MariaDB Database Cluster

Overview

The Skinny

Tungsten Clustering provides high availability, disaster recovery, and a host of other benefits for MySQL / MariaDB / Percona Server databases. In this blog post we will explore some of the shell aliases I use every day to administer various Tungsten Clusters.

Shell Aliases: A Quick Review

Quick and Easy

A shell alias is simply a way to create a shortcut for frequently-used command sequences.

For example, I like to shorten the command clear to cls, i.e.

If you create an alias on the fly it will be lost when the shell exits.

To save aliases so they are available to all shell sessions, update your shell’s profile or rc script.

For example, add the below line to the bottom of .bashrc, save and exit:

Open a new terminal window and confirm that your new alias works.

Shell Aliases: My Favorites

Speed and Efficiency

The aliases below are grouped together based on the functionality:

You can create your own – the sky is the limit!

Summary

The Wrap-Up

In this blog post we explored some of the shell aliases I use every day to administer various Tungsten Clusters.

To learn about Continuent solutions in general, check out https://www.continuent.com/solutions

---

The Library

Please read the docs!

For more information about Tungsten Cluster recovery procedures, please visit https://docs.continuent.com/tungsten-clustering-6.0/operations-recovery-master.html

Tungsten Clustering is the most flexible, performant global database layer available today – use it underlying your SaaS offering as a strong base upon which to grow your worldwide business!

For more information, please visit https://www.continuent.com/solutions

Want to learn more or run a POC? Contact us.

Categories: Mastering Tungsten Clustering, Monitor Tags: Aliases, bash, cli, Command, Command line, HA, High Availability, monitoring, mysql, Tungsten | Comments Off on Slick Command-Line Tricks for a Tungsten MySQL / MariaDB Database Cluster

Where are the logs for a Tungsten Cluster?

The Question

Recently, a customer asked us:

Where are the logs for a Tungsten Cluster and which are the proper log files to monitor if I do a master role switch to another node?

---

The Answer: Part I

Both basic and advanced logs are available!

The basic logs are symbolic links into the log subdirectory for each layer of the cluster.

For example, this is the listing of the default log directory, /opt/continuent/service_logs:

As you can see, each log file is a symlink to the user-level log for each of the layers, along with logs for backups.

You may also cd into each of the “real” log directories and look at the following, more detailed logs:

The Answer: Part II

Focus on the Manager and Connector logs

To monitor a master switch, the Manager log is best:
/opt/continuent/tungsten/tungsten-manager/log/tmsvc.log

and to a lesser degree the Connector log:
/opt/continuent/tungsten/tungsten-connector/log/connector.log

---

Summary

The Wrap-Up

In this blog post we discussed the Tungsten Cluster basic and advanced logs, their locations and which ones to use to monitor a master role switch to another node.

To learn about Continuent solutions in general, check out https://www.continuent.com/solutions

---

The Library

Please read the docs!

For more information about Tungsten clusters, please visit https://docs.continuent.com

Tungsten Clustering is the most flexible, performant global database layer available today – use it underlying your SaaS offering as a strong base upon which to grow your worldwide business!

For more information, please visit https://www.continuent.com/solutions

Want to learn more or run a POC? Contact us

Categories: Cluster, Manager, Mastering Tungsten Clustering, Monitor Tags: Clustering, Log, Logs, mysql, Network, Switch, Tungsten | Comments Off on Where are the logs for a Tungsten Cluster?