Bash: for loop alphabet

Source: For loop with Alphabet

[jerome@jeroboam] > for i in `seq 97 102`; do printf "\\$(printf %o $i)\n"; done
a
b
c
d
e
f

Linux: sum numbers

Source: Shell command to sum integers, one per line?

$ grep ussd\. MBQTUSG_20200302_001.csv | head -5
ussd.mt,8,02/03/2020 00:00:00,02/03/2020 23:59:59,88060
ussd.mo,576,02/03/2020 00:00:00,02/03/2020 23:59:59,172099
ussd.mt,576,02/03/2020 00:00:00,02/03/2020 23:59:59,172099
ussd.mo,1,02/03/2020 00:00:00,02/03/2020 23:59:59,254078
ussd.mt,34,02/03/2020 00:00:00,02/03/2020 23:59:59,254078

$ grep ussd\.m MBQTUSG_20200302_001.csv | cut -d, -f2 | perl -lpe '$c+=$_}{$_=$c'
243648

Format json file

The simplest way to reformat a one line JSON file into multiple lines is

jq . file.json

macOS: add PYTHONPATH to PostgreSQL brew service

Source: Changing the Homebrew Apache PATH variable.

Let's suppose we installed Python requests module.
On my system, requests has been installed in /usr/local/lib/python2.7/site-packages/.
Then, trying to import it from postgres like in

create or replace function tutu(o_status out int) as
$$
import sys, os
import requests, json
plpy.info(sys.version)
plpy.info(sys.path)
return (500)
$$ language plpython2u security definer;

will cause

select * from tutu()

to output

Query 1 ERROR: ERROR:  ImportError: No module named requests

By default, postgres will look in /Library/Python/2.7/site-packages/.
We then need to update PYTHONPATH.
With brew starting postgres, we will update the postgres plist file.

[jerome@jeroboam] > vi /usr/local/opt/postgresql\@9.4/homebrew.mxcl.postgresql\@9.4.plist

  KeepAlive
  
  Label
  homebrew.mxcl.postgresql@9.4
  EnvironmentVariables
  
    PYTHONPATH
    /usr/local/lib/python2.7/site-packages
  
  ProgramArguments
  
    /usr/local/opt/postgresql@9.4/bin/postgres
    -D
    /usr/local/var/postgresql@9.4
  
  RunAtLoad
  
  WorkingDirectory
  /usr/local
  StandardErrorPath
  /usr/local/var/log/postgresql@9.4.log

Then

[jerome@jeroboam] > brew services stop postgresql@9.4
[jerome@jeroboam] > brew services start postgresql@9.4

Then

select * from tutu()

outputs

INFO:  2.7.16 (default, Dec 13 2019, 18:00:32) 
[GCC 4.2.1 Compatible Apple LLVM 11.0.0 (clang-1100.0.32.4) (-macos10.15-objc-s
INFO:  ['/usr/local/lib/python2.7/site-packages', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', '/Users/jerome/Library/Python/2.7/lib/python/site-packages', '/Library/Python/2.7/site-packages', '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python', '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC']

macOS AnyConnect: AnyConnect cannot confirm it is connected to your secure gateway

Workaround: AnyConnect: Got an error after updating macOS Catalina

macOS: ld: warning: directory not found for option '-L/Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk/usr/lib'

If your PostgreSQL extension Makefile uses PGXS like in

PG_CONFIG = pg_config
...
PGXS := $(shell $(PG_CONFIG) --pgxs)
include $(PGXS)

And you updated MacOS since you installed PostgreSQL, you may experience the following error at compile time :

[jerome@jeroboam] > make
...
clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -D_XOPEN_SOURCE  -L/usr/local/Cellar/postgresql@9.4/9.4.18/lib -L/usr/local/opt/openssl/lib -L/usr/local/opt/readline/lib -L/Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk/usr/lib  -Wl,-dead_strip_dylibs   -bundle -bundle_loader /usr/local/Cellar/postgresql@9.4/9.4.18/bin/postgres -o src/pg_proctab.so src/pg_proctab.o
ld: warning: directory not found for option '-L/Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk/usr/lib'

Then replace MacOSX10.13 for CPPFLAGS and LDFLAGS in

[jerome@jeroboam] > sudo vi /usr/local/Cellar/postgresql\@9.4/9.4.18/lib/pgxs/src/Makefile.global

with e.g. MacOSX10.15.

MacOS: fatal error: 'sys/vfs.h' file not found

[jerome@jeroboam] > make
clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -D_XOPEN_SOURCE  -I. -I./ -I/usr/local/Cellar/postgresql@9.4/9.4.18/include/server -I/usr/local/Cellar/postgresql@9.4/9.4.18/include/internal -I/usr/local/opt/openssl/include -I/usr/local/opt/readline/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk/usr/include/libxml2   -c -o src/pg_proctab.o src/pg_proctab.c
src/pg_proctab.c:14:10: fatal error: 'sys/vfs.h' file not found
#include 
         ^~~~~~~~~~~
1 error generated.
make: *** [src/pg_proctab.o] Error 1

Then replace

#include <sys/vfs.h>

with

#if defined(__APPLE__)
#include <sys/mount.h>
#else
#include <sys/vfs.h>
#endif

byobu: name a new session

byobu new -s <session-name>

Clojure: if key exists: update, otherwise: assoc

Source : if key exists: update, otherwise: assoc

user> (update {:a [1]} :a (fnil conj []) 2)
{:a [1 2]}
user> (update {:a [1]} :b (fnil conj []) 2)
{:a [1], :b [2]}

macOS: install PHP on Catalina

Source: How to Install a PHP 7.2 on macOS 10.15 Catalina using Homebrew and PECL

macOS: know the details of running httpd

[jerome@jeroboam] > $(ps ax -o comm | grep -m 1 '[a]pache\|[h]ttpd') -V
Server version: Apache/2.4.41 (Unix)
Server built:   Oct  1 2019 10:31:38
Server's Module Magic Number: 20120211:88
Server loaded:  APR 1.7.0, APR-UTIL 1.6.1
Compiled using: APR 1.7.0, APR-UTIL 1.6.1
Architecture:   64-bit
Server MPM:     prefork
  threaded:     no
    forked:     yes (variable process count)
Server compiled with....
 -D APR_HAS_SENDFILE
 -D APR_HAS_MMAP
 -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
 -D APR_USE_SYSVSEM_SERIALIZE
 -D APR_USE_PTHREAD_SERIALIZE
 -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
 -D APR_HAS_OTHER_CHILD
 -D AP_HAVE_RELIABLE_PIPED_LOGS
 -D DYNAMIC_MODULE_LIMIT=256
 -D HTTPD_ROOT="/usr/local/Cellar/httpd/2.4.41_1"
 -D SUEXEC_BIN="/usr/local/opt/httpd/bin/suexec"
 -D DEFAULT_PIDLOG="/usr/local/var/run/httpd/httpd.pid"
 -D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
 -D DEFAULT_ERRORLOG="logs/error_log"
 -D AP_TYPES_CONFIG_FILE="/usr/local/etc/httpd/mime.types"
 -D SERVER_CONFIG_FILE="/usr/local/etc/httpd/httpd.conf"

macOS: where to find crash files.

On macOS, crashes of processes can be found in /Library/Logs/DiagnosticReports (system-wide) and ~/Library/Logs/DiagnosticReports (user).
These plain text files can be opened by the Console app or the System Reports app.

macOS: httpd: child pid 45333 exit signal Segmentation fault (11)

• macOS 10.15 (Catalina)
• php 7.3.10
• Apache 2.4.41

If you try to do SOAP requests with PHP 7 on macOS and you get permanent segmentation faults in the httpd error log, this is due to a bug in the PHP Apache module.
You can have an idea of the reasons of the segmentation fault in the .crash files automatically generated on macOS.

[jerome@jeroboam] > less /Users/jerome/Library/Logs/DiagnosticReports/httpd_2019-10-22-214827_jeroboam.crash

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_platform.dylib            0x00007fff71d21732 _platform_strlen + 18
1   libphp7.so                          0x000000010e5808e9 get_param + 56
2   libphp7.so                          0x000000010e581569 serialize_response_call2 + 648
3   libphp7.so                          0x000000010e57b371 serialize_response_call + 3807
4   libphp7.so                          0x000000010e57a01f zim_SoapServer_handle + 7188
5   libphp7.so                          0x000000010e739fa5 ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER + 412
6   libphp7.so                          0x000000010e709df9 execute_ex + 98
7   libphp7.so                          0x000000010e709f76 zend_execute + 319
8   libphp7.so                          0x000000010e6cfab7 zend_execute_scripts + 277
9   libphp7.so                          0x000000010e67b1a0 php_execute_script + 636
10  libphp7.so                          0x000000010e777f2f php_handler + 1092
11  httpd                               0x000000010dac6443 ap_run_handler + 51
12  httpd                               0x000000010dac69fd ap_invoke_handler + 240
13  httpd                               0x000000010dafc690 ap_internal_redirect + 54
14  mod_rewrite.so                      0x000000010dc895af handler_redirect + 128
15  httpd                               0x000000010dac6443 ap_run_handler + 51
16  httpd                               0x000000010dac69fd ap_invoke_handler + 240
17  httpd                               0x000000010dafc1cb ap_process_async_request + 864
18  httpd                               0x000000010dafc262 ap_process_request + 20
19  httpd                               0x000000010daf9078 ap_process_http_connection + 385
20  httpd                               0x000000010dad6c2c ap_run_process_connection + 51
21  mod_mpm_prefork.so                  0x000000010dc73807 child_main + 1085
22  mod_mpm_prefork.so                  0x000000010dc732c6 make_child + 409
23  mod_mpm_prefork.so                  0x000000010dc7330e startup_children + 72
24  mod_mpm_prefork.so                  0x000000010dc7258b prefork_run + 282
25  httpd                               0x000000010dad8f5f ap_run_mpm + 64
26  httpd                               0x000000010dacd58f main + 2119
27  libdyld.dylib                       0x00007fff71b23405 start + 1

The bug is known and referenced in Bug #73906.
A workaround is also proposed, which consists in having your PHP function return a hash map.
For example, if your function return an array of requestId + sessionAuthId, then replace it with

array('requestId' => $result[0], 'sessionAuthId' => $result[1])

.

postgreSQL: generate big data of unique numbers

Sources:

• How to generate random unique number in PostgreSQL using function
• How can I generate big data sample for Postgresql using generate_series and random?

CREATE OR REPLACE FUNCTION jerome_pseudo_encrypt(VALUE int) returns int AS $$
DECLARE
l1 int;
l2 int;
r1 int;
r2 int;
i int:=0;
BEGIN
 l1:= (VALUE >> 16) & 65535;
 r1:= VALUE & 65535;
 WHILE i < 3 LOOP
   l2 := r1;
   r2 := l1 # ((((1366 * r1 + 150889) % 714025) / 714025.0) * 32767)::int;
   l1 := l2;
   r1 := r2;
   i := i + 1;
 END LOOP;
 RETURN ((r1 << 16) + l1);
END;
$$ LANGUAGE plpgsql strict immutable;

create sequence jerome_seq maxvalue 2147483647;
create table prov_db.dt_hpe_dev (hpe_dev_id bigserial, sim_id bigint, constraint dt_hpe_dev_pkey primary key (hpe_dev_id));
insert into prov_db.dt_hpe_dev (sim_id) select jerome_pseudo_encrypt(nextval('jerome_seq')::int)  from generate_series(1, 1000000);

macOS: xcrun: error: invalid active developer path

Source: Git is not working after macOS Update

[jerome@jeroboam] > make
xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
[jerome@jeroboam] > xcode-select --install
xcode-select: note: install requested for command line developer tools

macOS: upgrade from Mojave 10.14.6 to Catalina 10.15

After some readings at macOS 10.15 Catalina is Now Available on the Mac App Store, followed these steps:

1. Upgraded Clover from 4920 to 5070 (macOS: from Clover 4920 to 5070)
2. Updated AppleALC.kext
3. Updated Lilu.kext
4. Updated WhateverGreen.kext
5. Followed the steps from [Guide] USB power property injection for Sierra (and later), and installed downloaded SSDT-EC.aml into EFI/CLOVER/ACPI/patched/
6. Rebooted
7. Downloaded and installed Catalina regularly.
   NB: At some point, the proper boot device was removed from the BIOS, but once reset, the install completed.
8. 

macOS: from Clover 4920 to 5070

Source: How to Update Clover v2.4k r4xxx to v2.5k r5xxx

1. Mount EFI partition

centurion:EFI jerome$ diskutil list
/dev/disk0 (internal, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *250.1 GB   disk0
   1:                        EFI EFI                     209.7 MB   disk0s1
   2:                  Apple_HFS hdd250                  249.7 GB   disk0s2

/dev/disk1 (internal, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *120.0 GB   disk1
   1:                        EFI EFI                     209.7 MB   disk1s1
   2:                 Apple_APFS Container disk2         119.8 GB   disk1s2

/dev/disk2 (synthesized):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      APFS Container Scheme -                      +119.8 GB   disk2
                                 Physical Store disk1s2
   1:                APFS Volume ssd120                  64.8 GB    disk2s1
   2:                APFS Volume Preboot                 183.4 MB   disk2s2
   3:                APFS Volume Recovery                2.0 GB     disk2s3
   4:                APFS Volume VM                      20.5 KB    disk2s4

/dev/disk3 (internal, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *500.1 GB   disk3
   1:                        EFI EFI                     209.7 MB   disk3s1
   2:                  Apple_HFS hdd500                  499.2 GB   disk3s2
   3:                 Apple_Boot Recovery HD             650.0 MB   disk3s3

centurion:EFI jerome$ sudo mkdir /Volumes/EFI
centurion:EFI jerome$ sudo mount_msdos /dev/disk1s1 /Volumes/EFI

2. Backup EFI partition

centurion:EFI jerome$ cp -r /Volumes/EFI/EFI /Volumes/ssd120/EFI-Backups/r4920/2019-09-20-20h34/

3. Remove /Volumes/EFI/EFI/
4. Start Clover installer | Customize
5. Match all the drivers from drivers64UEFI/
   
   
   
   
   
   As there is no match for AptioMemoryFix-64, leave it aside and complete the install.
   Contents of drivers/UEFI/ in the end:
   
6. Copy the old AptioMemoryFix-64 from EFI backup into the new drivers/UEFI/
7. Copy the kexts/Other/ contents from EFI backup into the new kexts/Other/
8. Copy themes/tonymacx86/ from EFI backup into the new themes/
9. Copy config.plist from EFI backup into the new CLOVER/
10. Copy ACPI/patched/SSDT-UIAC-ALL.aml from EFI backup into the new ACPI/patched/
11. Re-start

Clojure: remove from list

Source: How can I remove specific element from a list?
Tip: use a set.

;; Remove elements:
(remove #{3 5} [1 2 3 4 5 6 7 8 9])

;; Keep elements:
(keep #{7 5 3} [1 2 3 4 5 6 7 8 9])

;; Check if element exists:
(some #{5} [1 2 3 4 5 6 7 8 9])

Redshift epoch

Source: Redshift Epochs and Timestamps

select rating_date, extract('epoch' from rating_date::timestamp) as rating_date_epoch
from eu_db1.ft_rated_cdr
where extract('epoch' from cdr_date::timestamp) = 1561504138
and charged_co_id = 296661

PostgreSQL: distinct but only for one column

Source: Postgres: Distinct but only for one column
Problem: The following returns 2 distinct network_id for the same mccmnc_code.

with toto as (
    select mccmnc_code from (values (24024)) as x(mccmnc_code)
)
select dt_mccmnc.mccmnc_code, dt_mccmnc.network_id, dt_mccmnc.country_id
from toto
left join shared_db.dt_mccmnc 
    on dt_mccmnc.mccmnc_code = toto.mccmnc_code
    and dt_mccmnc.master_country_flg is true
    and localtimestamp(0) <@ dt_mccmnc.validity_period

-- mccmnc_code network_id country_id
-- 24024 1828 297
-- 24024 1816 297

So as to keep only one of the rows, use the following:

with toto as (
    select mccmnc_code from (values (24024)) as x(mccmnc_code)
)
select distinct on (dt_mccmnc.mccmnc_code) dt_mccmnc.mccmnc_code, dt_mccmnc.network_id, dt_mccmnc.country_id
from toto
left join shared_db.dt_mccmnc 
    on dt_mccmnc.mccmnc_code = toto.mccmnc_code
    and dt_mccmnc.master_country_flg is true
    and localtimestamp(0) <@ dt_mccmnc.validity_period

-- mccmnc_code network_id country_id
-- 24024 1828 297