byobu new -s <session-name>
A bozo's blog
Tuesday, November 26, 2019
Wednesday, November 13, 2019
Clojure: if key exists: update, otherwise: assoc
Source : if key exists: update, otherwise: assoc
user> (update {:a [1]} :a (fnil conj []) 2)
{:a [1 2]}
user> (update {:a [1]} :b (fnil conj []) 2)
{:a [1], :b [2]}
Thursday, October 31, 2019
Wednesday, October 23, 2019
macOS: know the details of running httpd
[jerome@jeroboam] > $(ps ax -o comm | grep -m 1 '[a]pache\|[h]ttpd') -V
Server version: Apache/2.4.41 (Unix)
Server built: Oct 1 2019 10:31:38
Server's Module Magic Number: 20120211:88
Server loaded: APR 1.7.0, APR-UTIL 1.6.1
Compiled using: APR 1.7.0, APR-UTIL 1.6.1
Architecture: 64-bit
Server MPM: prefork
threaded: no
forked: yes (variable process count)
Server compiled with....
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_SYSVSEM_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D DYNAMIC_MODULE_LIMIT=256
-D HTTPD_ROOT="/usr/local/Cellar/httpd/2.4.41_1"
-D SUEXEC_BIN="/usr/local/opt/httpd/bin/suexec"
-D DEFAULT_PIDLOG="/usr/local/var/run/httpd/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="/usr/local/etc/httpd/mime.types"
-D SERVER_CONFIG_FILE="/usr/local/etc/httpd/httpd.conf"
macOS: where to find crash files.
On macOS, crashes of processes can be found in /Library/Logs/DiagnosticReports (system-wide) and ~/Library/Logs/DiagnosticReports (user).
These plain text files can be opened by the Console app or the System Reports app.
These plain text files can be opened by the Console app or the System Reports app.
macOS: httpd: child pid 45333 exit signal Segmentation fault (11)
- macOS 10.15 (Catalina)
- php 7.3.10
- Apache 2.4.41
You can have an idea of the reasons of the segmentation fault in the .crash files automatically generated on macOS.
[jerome@jeroboam] > less /Users/jerome/Library/Logs/DiagnosticReports/httpd_2019-10-22-214827_jeroboam.crash
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 libsystem_platform.dylib 0x00007fff71d21732 _platform_strlen + 18 1 libphp7.so 0x000000010e5808e9 get_param + 56 2 libphp7.so 0x000000010e581569 serialize_response_call2 + 648 3 libphp7.so 0x000000010e57b371 serialize_response_call + 3807 4 libphp7.so 0x000000010e57a01f zim_SoapServer_handle + 7188 5 libphp7.so 0x000000010e739fa5 ZEND_DO_FCALL_SPEC_RETVAL_UNUSED_HANDLER + 412 6 libphp7.so 0x000000010e709df9 execute_ex + 98 7 libphp7.so 0x000000010e709f76 zend_execute + 319 8 libphp7.so 0x000000010e6cfab7 zend_execute_scripts + 277 9 libphp7.so 0x000000010e67b1a0 php_execute_script + 636 10 libphp7.so 0x000000010e777f2f php_handler + 1092 11 httpd 0x000000010dac6443 ap_run_handler + 51 12 httpd 0x000000010dac69fd ap_invoke_handler + 240 13 httpd 0x000000010dafc690 ap_internal_redirect + 54 14 mod_rewrite.so 0x000000010dc895af handler_redirect + 128 15 httpd 0x000000010dac6443 ap_run_handler + 51 16 httpd 0x000000010dac69fd ap_invoke_handler + 240 17 httpd 0x000000010dafc1cb ap_process_async_request + 864 18 httpd 0x000000010dafc262 ap_process_request + 20 19 httpd 0x000000010daf9078 ap_process_http_connection + 385 20 httpd 0x000000010dad6c2c ap_run_process_connection + 51 21 mod_mpm_prefork.so 0x000000010dc73807 child_main + 1085 22 mod_mpm_prefork.so 0x000000010dc732c6 make_child + 409 23 mod_mpm_prefork.so 0x000000010dc7330e startup_children + 72 24 mod_mpm_prefork.so 0x000000010dc7258b prefork_run + 282 25 httpd 0x000000010dad8f5f ap_run_mpm + 64 26 httpd 0x000000010dacd58f main + 2119 27 libdyld.dylib 0x00007fff71b23405 start + 1The bug is known and referenced in Bug #73906.
A workaround is also proposed, which consists in having your PHP function return a hash map. For example, if your function return an array of requestId + sessionAuthId, then replace it with
array('requestId' => $result[0], 'sessionAuthId' => $result[1])Friday, October 18, 2019
postgreSQL: generate big data of unique numbers
Sources:
- How to generate random unique number in PostgreSQL using function
- How can I generate big data sample for Postgresql using generate_series and random?
CREATE OR REPLACE FUNCTION jerome_pseudo_encrypt(VALUE int) returns int AS $$ DECLARE l1 int; l2 int; r1 int; r2 int; i int:=0; BEGIN l1:= (VALUE >> 16) & 65535; r1:= VALUE & 65535; WHILE i < 3 LOOP l2 := r1; r2 := l1 # ((((1366 * r1 + 150889) % 714025) / 714025.0) * 32767)::int; l1 := l2; r1 := r2; i := i + 1; END LOOP; RETURN ((r1 << 16) + l1); END; $$ LANGUAGE plpgsql strict immutable;
create sequence jerome_seq maxvalue 2147483647;
create table prov_db.dt_hpe_dev (hpe_dev_id bigserial, sim_id bigint, constraint dt_hpe_dev_pkey primary key (hpe_dev_id));
insert into prov_db.dt_hpe_dev (sim_id) select jerome_pseudo_encrypt(nextval('jerome_seq')::int) from generate_series(1, 1000000);
Wednesday, October 16, 2019
macOS: xcrun: error: invalid active developer path
Source: Git is not working after macOS Update
[jerome@jeroboam] > make xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun [jerome@jeroboam] > xcode-select --install xcode-select: note: install requested for command line developer tools
Saturday, October 12, 2019
macOS: upgrade from Mojave 10.14.6 to Catalina 10.15
After some readings at macOS 10.15 Catalina is Now Available on the Mac App Store, followed these steps:
- Upgraded Clover from 4920 to 5070 (macOS: from Clover 4920 to 5070)
- Updated AppleALC.kext
- Updated Lilu.kext
- Updated WhateverGreen.kext
- Followed the steps from [Guide] USB power property injection for Sierra (and later), and installed downloaded SSDT-EC.aml into EFI/CLOVER/ACPI/patched/
- Rebooted
- Downloaded and installed Catalina regularly.
NB: At some point, the proper boot device was removed from the BIOS, but once reset, the install completed. 
macOS: from Clover 4920 to 5070
Source: How to Update Clover v2.4k r4xxx to v2.5k r5xxx
- Mount EFI partition
- Backup EFI partition
- Remove /Volumes/EFI/EFI/
- Start Clover installer | Customize
- Match all the drivers from drivers64UEFI/
As there is no match for AptioMemoryFix-64, leave it aside and complete the install.
Contents of drivers/UEFI/ in the end:
- Copy the old AptioMemoryFix-64 from EFI backup into the new drivers/UEFI/
- Copy the kexts/Other/ contents from EFI backup into the new kexts/Other/
- Copy themes/tonymacx86/ from EFI backup into the new themes/
- Copy config.plist from EFI backup into the new CLOVER/
- Copy ACPI/patched/SSDT-UIAC-ALL.aml from EFI backup into the new ACPI/patched/
- Re-start
centurion:EFI jerome$ diskutil list
/dev/disk0 (internal, physical):
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *250.1 GB disk0
1: EFI EFI 209.7 MB disk0s1
2: Apple_HFS hdd250 249.7 GB disk0s2
/dev/disk1 (internal, physical):
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *120.0 GB disk1
1: EFI EFI 209.7 MB disk1s1
2: Apple_APFS Container disk2 119.8 GB disk1s2
/dev/disk2 (synthesized):
#: TYPE NAME SIZE IDENTIFIER
0: APFS Container Scheme - +119.8 GB disk2
Physical Store disk1s2
1: APFS Volume ssd120 64.8 GB disk2s1
2: APFS Volume Preboot 183.4 MB disk2s2
3: APFS Volume Recovery 2.0 GB disk2s3
4: APFS Volume VM 20.5 KB disk2s4
/dev/disk3 (internal, physical):
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *500.1 GB disk3
1: EFI EFI 209.7 MB disk3s1
2: Apple_HFS hdd500 499.2 GB disk3s2
3: Apple_Boot Recovery HD 650.0 MB disk3s3
centurion:EFI jerome$ sudo mkdir /Volumes/EFI
centurion:EFI jerome$ sudo mount_msdos /dev/disk1s1 /Volumes/EFIcenturion:EFI jerome$ cp -r /Volumes/EFI/EFI /Volumes/ssd120/EFI-Backups/r4920/2019-09-20-20h34/
Wednesday, September 25, 2019
Clojure: remove from list
Source: How can I remove specific element from a list?
Tip: use a set.
Tip: use a set.
;; Remove elements:
(remove #{3 5} [1 2 3 4 5 6 7 8 9])
;; Keep elements:
(keep #{7 5 3} [1 2 3 4 5 6 7 8 9])
;; Check if element exists:
(some #{5} [1 2 3 4 5 6 7 8 9])
Redshift epoch
Source: Redshift Epochs and Timestamps
select rating_date, extract('epoch' from rating_date::timestamp) as rating_date_epoch
from eu_db1.ft_rated_cdr
where extract('epoch' from cdr_date::timestamp) = 1561504138
and charged_co_id = 296661
Wednesday, September 4, 2019
PostgreSQL: distinct but only for one column
Source: Postgres: Distinct but only for one column
Problem: The following returns 2 distinct network_id for the same mccmnc_code.
Problem: The following returns 2 distinct network_id for the same mccmnc_code.
with toto as (
select mccmnc_code from (values (24024)) as x(mccmnc_code)
)
select dt_mccmnc.mccmnc_code, dt_mccmnc.network_id, dt_mccmnc.country_id
from toto
left join shared_db.dt_mccmnc
on dt_mccmnc.mccmnc_code = toto.mccmnc_code
and dt_mccmnc.master_country_flg is true
and localtimestamp(0) <@ dt_mccmnc.validity_period
-- mccmnc_code network_id country_id
-- 24024 1828 297
-- 24024 1816 297
So as to keep only one of the rows, use the following:
with toto as (
select mccmnc_code from (values (24024)) as x(mccmnc_code)
)
select distinct on (dt_mccmnc.mccmnc_code) dt_mccmnc.mccmnc_code, dt_mccmnc.network_id, dt_mccmnc.country_id
from toto
left join shared_db.dt_mccmnc
on dt_mccmnc.mccmnc_code = toto.mccmnc_code
and dt_mccmnc.master_country_flg is true
and localtimestamp(0) <@ dt_mccmnc.validity_period
-- mccmnc_code network_id country_id
-- 24024 1828 297
Monday, August 26, 2019
PostgreSQL: create table from CSV
Import CSV data into new table.
create table shared_db.dt_ranking (ranking_id bigserial, vers text,tadig_code text ,mcc text ,mnc text ,advanced int, lite int, vplmn_rank int, constraint dt_ranking_pkey primary key (ranking_id) ) \copy shared_db.dt_ranking (vers,tadig_code,mcc,mnc,advanced,lite,vplmn_rank) from '/tmp/ranking.csv' with (format csv, delimiter ',', header true);
Wednesday, June 26, 2019
Clojure: nested destructuring
user> toto
{:pim 1, :pam {:poum [{:pif 1, :paf 2}]}}
user> (let [{:keys [pim]
{:keys [poum] [{:keys [pif paf]}] :poum} :pam} toto]
{:pim pim :poum poum :pif pif :paf paf})
{:pim 1, :poum [{:pif 1, :paf 2}], :pif 1, :paf 2}
Tuesday, May 21, 2019
macOS: access hidden user Library in Time Machine
- Enter Time Machine
- Type Cmd Shift G
- Input ~/Library
Tuesday, April 23, 2019
postgreSQL: get some table statistics
select * from pg_stat_all_tables where schemaname = 'short_message_db' and relname = 'dt_msg'; select * from pg_statio_all_tables where schemaname = 'short_message_db' and relname = 'dt_msg';
Wednesday, April 3, 2019
macOS: bash error: declare: -A: invalid option
[jerome@jeroboam] > ./avws-stores.sh
+++ dirname ./avws-stores.sh
++ cd .
++ pwd
+ SCRIPT_DIR=/Users/jerome/src/boss-sms-vas/release
+ REPO_NAME=boss-sms-vas-stores
+ declare -A AWS_S3_BUCKETS
./avws-stores.sh: line 16: declare: -A: invalid option
declare: usage: declare [-afFirtx] [-p] [name[=value] ...]
Source: Upgrading Bash on macOS
[jerome@jeroboam] > brew install bash
[jerome@jeroboam] > sudo vi /etc/shells
/bin/bash /bin/csh /bin/ksh /bin/sh /bin/tcsh /bin/zsh /usr/local/bin/bash
[jerome@jeroboam] > chsh -s /usr/local/bin/bash
AWS: list Aurora supported PostgreSQL versions
[jerome@jeroboam] > aws rds describe-db-engine-versions --engine aurora-postgresql --region eu-west-1
[
...
{
"Engine": "aurora-postgresql",
"DBParameterGroupFamily": "aurora-postgresql10",
"SupportsLogExportsToCloudwatchLogs": false,
"SupportsReadReplica": false,
"DBEngineDescription": "Aurora (PostgreSQL)",
"SupportedEngineModes": [
"provisioned"
],
"EngineVersion": "10.6",
"DBEngineVersionDescription": "Aurora PostgreSQL (compatible with PostgreSQL 10.6)",
"ValidUpgradeTarget": []
}
]
Monday, April 1, 2019
Clojure spec: key required depending on another key value
;; status > 0 => msgid required
(defmulti sendmt-rsp-mm (fn [rsp] (-> rsp :status (clojure.string/starts-with? "-")))) ; Actually means: status < 0
(defmethod sendmt-rsp-mm false [_] (s/keys :req-un [::status
::msgid]))
(defmethod sendmt-rsp-mm true [_] (s/keys :req-un [::status
::text]))
(s/def ::sendmt-rsp (s/multi-spec sendmt-rsp-mm (fn [gen-v _] gen-v)))
or
(s/def ::sendmt-rsp (s/and (s/keys :req-un [::status])
#(if (clojure.string/starts-with? (:status %) "-")
(contains? % :text)
(contains? % :msgid))))
user> (s/valid? ::sendmt-rsp {:status "2" :text "caca"})
false
user> (s/valid? ::sendmt-rsp {:status "2" :msgid "caca"})
true
user> (s/valid? ::sendmt-rsp {:status "-2" :text "caca"})
true
user> (s/valid? ::sendmt-rsp {:status "-2" :msgid "caca"})
false
Friday, March 22, 2019
postgreSQL: collapse contiguous ranges
Source: Efficiently select beginning and end of multiple contiguous ranges in Postgresql query
Or if we do not have a "last" column.
The problem is that with recursive tends to be slow when dealing with millions of rows.
How to find the boundaries of groups of contiguous sequential numbers? proposes a faster approach.
with recursive
data as (
select * from (
values ('foo', 2, 3),
('foo', 3, 4),
('foo', 4, 5),
('foo', 10, 11),
('foo', 11, 13),
('foo', 13, 15),
('bar', 1, 2),
('bar', 2, 4),
('bar', 7, 8)
) as baz (name, first, last)
),
recur (name, first, last) as (
select name, first, last, last-first as span from data
union all
select name, data.first, data.last, recur.span+data.last-data.first as span
from data join recur using (name)
where data.first = recur.last
)
select name, start, start + span as end, span from (
select name, (last-span) as start, max(span) as span from (
select name, first, last, max(span) as span
from recur
group by name, first, last
) as z
group by name, (last-span)
) as z
┌──────┬───────┬─────┬──────┐
│ name │ start │ end │ span │
├──────┼───────┼─────┼──────┤
│ bar │ 1 │ 4 │ 3 │
│ bar │ 7 │ 8 │ 1 │
│ foo │ 10 │ 15 │ 5 │
│ foo │ 2 │ 5 │ 3 │
└──────┴───────┴─────┴──────┘
(4 rows)
Or if we do not have a "last" column.
with recursive
data as (
select * from (
values ('foo', 2),
('foo', 3),
('foo', 4),
('foo', 10),
('foo', 11),
('foo', 13),
('bar', 1),
('bar', 2),
('bar', 7)
) as baz (name, first)
),
recur (name, first) as (
select name, first, 1 as span from data
union all
select name, data.first, recur.span+1 as span
from data
join recur using (name)
where data.first = recur.first + 1
)
select name, start, start + span - 1 as end, span from (
select name, (first+1-span) as start, max(span) as span from (
select name, first, max(span) as span
from recur
group by name, first
) as z
group by name, start
) as z
order by name, start
┌──────┬───────┬─────┬──────┐
│ name │ start │ end │ span │
├──────┼───────┼─────┼──────┤
│ bar │ 1 │ 2 │ 2 │
│ bar │ 7 │ 7 │ 1 │
│ foo │ 2 │ 4 │ 3 │
│ foo │ 10 │ 11 │ 2 │
│ foo │ 13 │ 13 │ 1 │
└──────┴───────┴─────┴──────┘
(5 rows)
The problem is that with recursive tends to be slow when dealing with millions of rows.
How to find the boundaries of groups of contiguous sequential numbers? proposes a faster approach.
with
data as (
select * from (
values ('foo', 2),
('foo', 3),
('foo', 4),
('foo', 10),
('foo', 11),
('foo', 13),
('bar', 1),
('bar', 2),
('bar', 7)
) as baz (name, first)
),
island as (
select first - row_number() over (order by name, first) as grp,
name, first
from data
)
select name, min(first) as start, max(first) as end
from island
group by name, grp
order by name, start
┌──────┬───────┬─────┐
│ name │ start │ end │
├──────┼───────┼─────┤
│ bar │ 1 │ 2 │
│ bar │ 7 │ 7 │
│ foo │ 2 │ 4 │
│ foo │ 10 │ 11 │
│ foo │ 13 │ 13 │
└──────┴───────┴─────┘
(5 rows)
Thursday, March 14, 2019
macOS: terminal UTF-8 issues
If you cannot display unicode characters in psql, e.g.
[jerome@jeroboam] > psql mydb
mydb=# \d public.tap_funky
View "public.tap_funky"
<E2><94><8C><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><AC><E2><94><80><E2><94><80><E2>
<94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><AC><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94>
<80><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><80><E2><94><90>
<E2><94><82> Column <E2><94><82> Type <E2><94><82> Modifiers <E2><94><82>
Then
[jerome@jeroboam] > vi ~/.bash_profileexport LANG=en_US.UTF-8
[jerome@jeroboam] > source ~/.bash_profilemydb=# \d public.tap_funky
View "public.tap_funky"
┌─────────────┬──────────────┬───────────┐
│ Column │ Type │ Modifiers │
├─────────────┼──────────────┼───────────┤
│ oid │ oid │ │
│ schema │ name │ │
│ name │ name │ │
│ owner │ name │ │
│ args │ text │ │
│ returns │ text │ │
│ langoid │ oid │ │
│ is_strict │ boolean │ │
│ is_agg │ boolean │ │
│ is_definer │ boolean │ │
│ returns_set │ boolean │ │
│ volatility │ character(1) │ │
│ is_visible │ boolean │ │
└─────────────┴──────────────┴───────────┘
macOS: Library not loaded: /usr/local/opt/readline/lib/libreadline.7.dylib
If you get:
[jerome@jeroboam] > psql mydb
dyld: Library not loaded: /usr/local/opt/readline/lib/libreadline.7.dylib
Referenced from: /usr/local/bin/psql
Reason: image not found
Abort trap: 6
[jerome@jeroboam] > ln -s /usr/local/opt/readline/lib/libreadline.8.0.dylib /usr/local/opt/readline/lib/libreadline.7.dylib
Monday, March 4, 2019
AWS MFA with Yubikey and macOS
Purpose: Take advantage of the Yubikey to generate the 6 digit TOTP required by AWS MFA without using Google Authenticator.
Downside: You can't just press the Yubikey button and have the code generated as you would expect, but this can be alleviated with a keyboard shortcut.
Sources:
but macOS Mojave, Automator “Not authorized to send Apple events to System Events.” gave the solution:
I quote:
Downside: You can't just press the Yubikey button and have the code generated as you would expect, but this can be alleviated with a keyboard shortcut.
Sources:
- Use a YubiKey as a MFA device to replace Google Authenticator
- Faster AWS/PayPal/TOTP two factor auth with Yubikey
- Install the Yubikey CLI
- Insert the Yubikey
- Show a list of configured TOTP accounts
- Log in to AWS Management Console as usual, pop up the menu by clicking on your user name and select My Security Credentials.
- Push the "Manage MFA Device" button.
- Select Remove to disable MFA, and then re-start the procedure to activate MFA again.
- In the next screen, select “Virtual MFA device”.
- Show the secret key: it will be passed to ykman.
- Configure MFA for your service.
- Then, this will get you a 6 digit code.
- Start Automator, and create a new Quick Action.
- Search for applescript.
- Drag and drop "Run applescript" to the right hand side, select "Workflow receives no input", and type the following code.
- Then File | Save, and go to System Preferences | Keyboard | Shortcuts | Services to assign a shortcut to your new service.
- Now, simply log in the AWS Management Console with your password, and when the site asks for the MFA, use the programmed shortcut, which will automatically generate the 6 digit code and grant you access.
brew install ykman[jerome@jeroboam] > ykman oath list
[jerome@jeroboam] > 
[jerome@jeroboam] > ykman oath add 'Amazon Web Services:toto@org-prod' [jerome@jeroboam] > ykman oath code --single 'Amazon Web Services:toto@org-prod'
but macOS Mojave, Automator “Not authorized to send Apple events to System Events.” gave the solution:
I quote:
System Preferences > Security & Privacy > Accessibility > Click Automator and TADA it works.End quote
Wednesday, January 9, 2019
macOS: merge 2 terminals in tabs
Source: How can I merge two terminal windows in OS X lion?
This still actually works with macOS 10.14.2
This still actually works with macOS 10.14.2
Subscribe to:
Posts (Atom)