Hi Monks,

I'm doing what most Americans are: not watching tv and blasting 80's music so that I can't hear. I've had to develop some software tools related to downloading social media posts to follow a guy who has become a nemesis of sorts, a cat named Tim Ballard. This guy is problematic and disgusting in many different ways. He puts out this mega-wave of bogus stuff about his own supposed heroism when he's a demonstrable affinity fraud who claims to be this Great Liberator of child sex slaves (zero proof). So the topic is supergross on top of that, and not something I want on my machine. So a big part of this software is conditioning a USB target drive to download and watermark it with a time and source stamp for eventual use by a small village of female complainants whom he defames and threatens in these social media posts.

The software is a disheveled pile of scripts at this point, and I was hoping to bring it to perl to be able to dress it up with the trappings of a distro, where I could then do more testing as I go for other types of these social media posts. I will say that I did give Leon the 11 bucks for a month of twitter prime to be able to access the API and grok, and was never able to do the former, while the latter was available for one week, but then I could only get grok2 and if I subscribed to prime, which is a maddening bad-logic game. As far as I can tell, Leon has just transformed that company into his version of Truth Social. (I actually bought stock in DJT so that I could be part of shareholders' lawsuit if the bottom should drop out while an officer thereof makes false representations.)

Let's get to the software. I'll have a handful of bash scripts that are kind of working, and since there's failures, my victories come on the heels of meticulous logging. I like to wrap bash scripts so get that failure point. Here, it's trying to get this to github. Here's the wrapper:

#!/usr/bin/perl 

# /media/fritz/175F-DC61/2022.bak/2.22.bak
 
use v5.030;
use Time::Piece;
use Log::Log4perl;
use IPC::System::Simple qw/systemx capturex/;
use utf8;



my ($argv1) = @ARGV;
if (not defined $argv1) {
  die "Need argv1\n";
}

my $t         = localtime;
my $jd        = $t->julian_day;
my $log_conf4 = '/home/fritz/Documents/perlmonks/conf_files/4.conf';
Log::Log4perl::init($log_conf4);    #info
my $logger = Log::Log4perl->get_logger();
$logger->info("Time is $t");
$logger->info("Julian day is $jd");
$logger->info("$0");

my $ini_path = qw( /home/fritz/Documents/perlmonks/conf_files/1.&#1090
+;айный.txt );

my $ref_config = get_тайный($ini_p
+ath);
#$DB::single = 1;
my %h = %$ref_config;

my $capture = capturex "bash","6.git.sh", "@ARGV";
$logger->info("capx: $capture");
sub get_тайный {

  use Config::Tiny;
  use Data::Dump;
  my %h;    #creating here and exporting reference to caller

  my $ini_path  = shift;               #caller provides inipath
  my $sub_hash1 = "gitlab";
  my $Config    = Config::Tiny->new;

  $Config = Config::Tiny->read( $ini_path, 'utf8' );

  # -> is optional between brackets
  $h{email} = $Config->{$sub_hash1}{'email'};
  $h{name}  = $Config->{$sub_hash1}{'name'};
  $h{pass}  = $Config->{$sub_hash1}{'pass'};

  my $sub_hash2 = "bash";
  $h{bash_root} = $Config->{$sub_hash2}{'bash_root'};
  my $ref_config = \%h;
  #dd $ref_config;
  $DB::single = 1;
  return ($ref_config);
}

__END__
[download]

This has sat on mothballs since the last time that it worked. I managed to lock myself out of my git account by cutting the cord with my iphone, having initiated 2FA haphazardly (no recovery codes). It was all I could do to tell my siblings that I was leaving my iphone, so I didn't think of gitlab. Anyways, I've decided I don't need 2FA for this, and have a new account that I think is configured correctly, but my terminal doesn't want to agree with me. It's wrapping this bash script that's me trying to tap my way through git push/pull/punch...

#!/bin/bash

# Create README.md with the current directory information
pwd > README.md

# Initialize a new Git repository (or reinitialize if it already exist
+s)
git init

# Hardcode Git user information
git config --global user.email "redacted@redacted"
git config --global user.name "@redacted"

# Add a list of files to README.md
ls >> README.md

# Add files to the staging area
git add README.md
git add *.pl *.sh *.txt *.py

# Set up the remote repository URL (using SSH)
if ! git remote | grep -q origin; then
    git remote add origin git@gitlab.com:aardvark/hja.git
else
    echo "Remote 'origin' already exists."
fi

# Create a verbose, descriptive commit message based on staged changes
who=$(printenv USER)
now=$(date)
file_list=$(git diff --cached --name-only | tr '\n' ', ' | sed 's/, $/
+/')

# Commit all staged changes
git commit -m "On $now, $who added changes in files: $file_list" || { 
+echo "Failed to commit changes"; exit 1; }

# Push changes to the main branch
git branch -M main
git push -u origin main

echo "Script completed successfully."
[download]

I get good partial results:

<c>2024/11/05 19:36:08 INFO Time is Tue Nov  5 19:36:07 2024 
2024/11/05 19:36:08 INFO Julian day is 2460620.60841435 
2024/11/05 19:36:08 INFO 4.wrap.pl 
2024/11/05 19:36:13 INFO capx: Reinitialized existing Git repository i
+n /home/fritz/Desktop/dl2/repack/.git/
Remote 'origin' already exists.
[main 8f1999b] On Tue 05 Nov 2024 07:36:08 PM MST, fritz added changes
+ in files: 4.wrap.pl,
 1 file changed, 1 insertion(+), 1 deletion(-)
Script completed successfully.
[download]

Q1) How do I get this onto gitlab?

Why re-invent the download wheel? I needed a lot more forensics on the video than I was gonna get from somebody's else's dl app. This is typical output; winnowing and filtering the metadata is also critical to not getting buried on STDOUT. I'm particularly pleased that this gives me a machine-state. I endeavor to leave every step like this as a way for the envelope to persist.

This is typical .json output that accompanies the dl.

{
    "download_path": "/media/fritz/9766-DD0B/2024-11-03/",
    "cookie_path": null,
    "url": "https://www.youtube.com/watch?v=YDP2tR6MXNA",
    "font": "Arial Bold",
    "font_size": 64,
    "username_color": "yellow",
    "date_color": "cyan",
    "timestamp_color": "red",
    "username_position": [
        "left",
        "top"
    ],
    "date_position": [
        "left",
        "bottom"
    ],
    "timestamp_position": [
        "right",
        "bottom"
    ],
    "overall_start": 2,
    "caption_top": "15%",
    "caption_bottom": "75%",
    "line_width": "8%",
    "hor_offset": "4%",
    "cap_length": 5,
    "max_number": 60,
    "max_char_width": 65,
    "next_line": 1.7,
    "pause_between_para": 2,
    "source_path": "/app/data/4.source.txt",
    "shadow": {
        "color": "black",
        "offset": 5,
        "opacity": 0.6
    },
    "video_title": "Tim_Ballard_20241028_4_watermarked_captioned",
    "video_date": "20241029",
    "uploader": "Perl Gonzalez",
    "duration": 151,
    "width": 1920,
    "height": 1080,
    "id": "YDP2tR6MXNA",
    "ext": "mkv",
    "resolution": "1920x1080",
    "fps": 24,
    "tbr": 2183.114,
    "protocol": "https+https",
    "vcodec": "avc1.640028",
    "vbr": 2065.741,
    "acodec": "opus",
    "abr": 117.373,
    "asr": 48000,
    "original_filename": "/media/fritz/9766-DD0B/2024-11-03/Perl_Gonza
+lez_20241029.mkv",
    "to_process": "/media/fritz/9766-DD0B/2024-11-03/Perl_Gonzalez_202
+41029.mkv"
}
[download]

One thing I have to remark on is that I'm not using a cookie to make this happen, which is surprising, because it still happens. (OT: The python library used is yt_dlp for anyone who wants to enlighten me.)

Central to this is a Dockerfile:

# Start with the base image (Ubuntu in this case)
FROM ubuntu:22.04

# Update apt repository and install necessary dependencies including f
+fmpeg
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
    ffmpeg \
    python3 \
    python3-pip && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# Set the working directory
WORKDIR /app

# Copy all files into the Docker image
COPY . /app

# Install Python dependencies
RUN pip3 install --upgrade pip && pip3 install -r requirements.txt --r
+oot-user-action=ignore

RUN ls -la * /app
[download]

This has become simpler as the bash wrapper gets fancier. It's evolved away the CMD at the bottom; this is how docker gets invoked from the bash wrapper:

# Check if the Docker image exists
if [[ "$(docker images -q my_dl:latest 2> /dev/null)" == "" ]]; then
    echo "Docker image 'my_dl:latest' does not exist. Building the ima
+ge..."
    docker build -t my_dl .
else
    echo "Docker image 'my_dl:latest' found."
fi

# Proceed with Docker run command using the validated paths
echo "Running Docker with the specified volume and config path..."
docker run --rm \
  -v "$(pwd)":/app \
  -v "$Config_Path":/app/data/8.config.json \
  -v "$usb_mount_point":"$usb_mount_point" \
  my_dl python3 4.caller.py "$1" | tee -a "$LOGFILE"
[download]

The -v is for volumes, and it's a clever machine who figured this part out rather than me.

What do I want to do with this? I want perl to pull the wagon from dl to watermark to postprocess. Watermark example of Mr. Disgusting. I've got the units written but not combined, but I want it to be with the structure of the distro, where there's a place to put a bunch of oddball scripts and keep data segregated. Glad for any suggestions.

With the help of a superintelligent chatbot, I can wade through the python, but what I really miss in assembling programming units is the perl debugger. What I'm finding is that I get a lot of functionality out of the docker debugger, but I haven't found a python equivalent. It could well exist. In some sense, I want to bring this back to my own turf. And it occurs to me that if I'm going to embellish what's going on with the Dockerfile, then I want to do it with perl because it's native. Again, accepting anybody's experience in doing so.

Alright, well I'm glad I did something other than watch the boobtube and fret tonight. Hope this find you all well.

Gruss aus Amiland,

"nauseous optimism"