Miscellaneous - Random knowledge that won't fit any other category

• Perl regular expressions
• Perl Regular Expressions Reference
• Perl regular expressions quick start

\K

causes the regex engine to "keep" (hold) everything it had matched prior to the \K and not include it in $&

• the documentation says both are equivalent :

  echo foobar | perl -pe 's/(foo)bar/$1/g;'
  echo foobar | perl -pe 's/foo\Kbar//g;'	but this one is much more efficient

  and return :
  foo

• I still have to figure how the regex is processed :
  
  echo foobar | perl -pe 's/foo\Kbar//g;'
  foo
  
  - get the 'foobar' input string
  - read the 'foo\Kbar' regex as the "search" part of the 's///' command
  - because of the \K, it "holds" (forgets ?) anything matched before : i.e. the 'foo' part
  - remains the 'bar' part
  - the 's///' command becomes : 's/bar//', i.e. "remove 'bar'"
  
  
  echo 'foobar' | grep -oP 'foo\Kbar'
  bar
  
  - looks like it does the opposite of the example above, but it's not ;-)
  - 'foo\Kbar' can be understood as "forget 'foo', keep 'bar'"
  - in the example above, this is provided as the "search" part of an 's///' command : replace 'bar' with nothing, i.e. keep 'foo'
  - in this example, it is what's happening : grep [options] ""forget 'foo', keep 'bar'"" : outputs 'bar'
  

Extract data from XML

<data>
	<key name="foo">
		<value>ga bu</value>
	</key>
	<key name="bar">
		<value>zo meu</value>
	</key>
</data>

cat << EOF | sed -r -e 's/</\n</g' -e 's/>/>\n/g' | sed -r '/(^[[:blank:]]*$|[<>])/ d'
<data>
	<key name="foo">
		<value>ga bu</value>
	</key>
	<key name="bar">
		<value>zo meu</value>
	</key>
</data>
EOF

ga bu
zo meu

cat << EOF | grep -oP '<value>\K[^<]+'
<data>
	<key name="foo">
		<value>ga bu</value>
	</key>
	<key name="bar">
		<value>zo meu</value>
	</key>
</data>
EOF

ga bu
zo meu

The magic here is that '<value>\K[^<]+' is interpreted as :
- focus on what comes after <value>
- match any sequence of characters until you find a <

basic vs unspecified format

• basic : has a minimal number of separators
• unspecified (aka extended) format : has separators to enhance human readability
  • - between date values
  • : between time values

T between date and time values

serves as a delimiter between date and time when they are concatenated

Trailing Z

indicates that the time is in UTC

What about leap seconds ?

Which one should I use ?

• change display graphics
• control cursor movement
• reassign keys

text color

ansiCode_start='\033['
ansiCode_stop='m'
black="${ansiCode_start}0;30${ansiCode_stop}"
red="${ansiCode_start}0;31${ansiCode_stop}"
green="${ansiCode_start}0;32${ansiCode_stop}"
orange="${ansiCode_start}0;33${ansiCode_stop}"
blue="${ansiCode_start}0;34${ansiCode_stop}"
purple="${ansiCode_start}0;35${ansiCode_stop}"
cyan="${ansiCode_start}0;36${ansiCode_stop}"
lightGray="${ansiCode_start}0;37${ansiCode_stop}"
darkGray="${ansiCode_start}1;30${ansiCode_stop}"
lightRed="${ansiCode_start}1;31${ansiCode_stop}"
lightGreen="${ansiCode_start}1;32${ansiCode_stop}"
yellow="${ansiCode_start}1;33${ansiCode_stop}"
lightBlue="${ansiCode_start}1;34${ansiCode_stop}"
lightPurple="${ansiCode_start}1;35${ansiCode_stop}"
lightCyan="${ansiCode_start}1;36${ansiCode_stop}"
white="${ansiCode_start}1;37${ansiCode_stop}"
noColor="${ansiCode_start}0${ansiCode_stop}"

printf "I ${red}<3${noColor} Linux\n"
echo -e "I ${red}<3${noColor} Linux"

for color in black red green orange blue purple cyan lightGray darkGray lightRed lightGreen yellow lightBlue lightPurple lightCyan white; do echo -e "${!color}${color}${noColor}"; done

text decoration

echo -e "\033[0;31m___solid_________\033[0m"
echo -e "\033[1;31m___bold__________\033[0m"
echo -e "\033[2;31m___dark__________\033[0m"
echo -e "\033[3;31m___italic________\033[0m"
echo -e "\033[4;31m___underlined____\033[0m"
echo -e "\033[5;31m___blink slow____\033[0m"
echo -e "\033[6;31m___blink fast____\033[0m"
echo -e "\033[7;31m___reverse_______\033[0m"
echo -e "\033[8;31m___hide__________\033[0m"
echo -e "\033[9;31m___strikeout_____\033[0m"

background color

echo -e "\033[41m___on red______\033[0m \t \033[101m___on high intensity red______\033[0m"
echo -e "\033[42m___on green____\033[0m \t \033[102m___on high intensity green____\033[0m"
echo -e "\033[43m___on yellow___\033[0m \t \033[103m___on high intensity yellow___\033[0m"
echo -e "\033[44m___on blue_____\033[0m \t \033[104m___on high intensity blue_____\033[0m"

high intensity (i.e. same color than the "bright", but not "bold")

echo -e "\033[0;34m___solid_________________\033[0m"
echo -e "\033[1;34m___bold__________________\033[0m"
echo -e "\033[0;94m___high intensity________\033[0m"
echo -e "\033[1;94m___bold high intensity___\033[0m"

combinations

echo -e "\033[43m\033[1;31m___bright red on yellow_____\033[0m"
echo -e "\033[43m\033[5;31m___blinking red on yellow___\033[0m"

x=123; y=456; echo "BEFORE : x=$x, y=$y"; x=$((x-y)); y=$((x+y)); x=$((y-x)); echo "AFTER : x=$x, y=$y"

• basic regular expressions (BRE)
• extended regular expressions (ERE)
• Perl regular expressions

• characters such as ?, +, {, |, ( and ) have no special meaning
• they must be escaped with \ to become special

Are we using BRE or ERE ?

echo -e "a\nb\nc" | /bin/grep 'b+'
(nothing)					the + is not special : BRE

echo -e "a\nb\nc" | /bin/grep 'b\+'
b						the + has become "special" : ERE

echo -e "a\nb\nc" | /bin/grep -E 'b+'
b						we've explicitly requested ERE with grep -E

[[ "foo" =~ ^f.o$ ]] && echo match || echo 'no match'
match								. matches a single character (o, ., )

[[ "foo" =~ ^f\.o$ ]] && echo match || echo 'no match'
no match							\. does NOT match any single character

[[ "f.o" =~ ^f\.o$ ]] && echo match || echo 'no match'
match								\. matches a ., showing that =~ uses ERE

• View full list on RegExLib.com
• View reference on regular-expressions.info :
  • basic
  • advanced

y = ax + b

y1 = ax1 + b
y2 = ax2 + b

b = y1 - ax1

y2 = ax2 + y1 - ax1
y2 = a(x2 - x1) + y1

a(x2 - x1) = y2 - y1

a = ( y2 - y1 ) / ( x2 - x1 )

y1 = ax1 + b

b = y1 - ax1

b = y1 - x1 * ( y2 - y1 ) / ( x2 - x1 )

y = ax + b

ax + b = n
ax = n - b

x = ( n - b ) / a

• Would you mind looking at my luggage just for a minute ? (Then stop a thief)
• What time is it, please ? (...) Do you have a quarter, I need to make an important phone call ?

• Even though the movie is bad, you watch it until the end because you PAID for it.
• Drinking a bad wine just because it is a famous (and expensive) bottle.

• deciding it with a "STOP" decision. But such traps work so well because of the freeze effect.
• setting a stop threshold before entering such a trap : wait n minutes only, spend at most n €, ...

• The bus is late. Miss a first taxi, believing the bus will arrive soon. Miss another taxi. No bus. No more taxi. Walk back home. Waiting + walking was longer than taking the 1st taxi, or walk without waiting.
• The Vietnam war, for the USA.

1. Just ask ! Sounds stupid, but this often works
   • between colleagues in a rush
   • before someone leaves for holidays
   • with the appropriate social engineering methods
2. Guess !
   • Birthday date, kid's 1^st name, pet name, car plate number, loved person nickname, favorite band / song / movie / football team / ice cream brand /
   • Consider the most famous passwords in the world : love, secret, god, password, 123456, qwerty, letmein, 123123, ... (list)
3. Steal it !
   • It may be noted on a Post-it™ note under the keyboard
   • or stored in a passwords.xls on a USB key
   • You can even —subtly— ask your victim about his mother's maiden name / childhood pet name / favorite teacher's name... or any question used in a Lost password form
4. Break it with a dictionary attack.
5. Break it with a brute force attack.

Their methods

• No password is safe forever :
  • make it an habit of changing passwords before they become crackable (i.e. if a given password is assumed to take 2 years to be brute-forced, change it after 1 year : whenever it's cracked, you changed it already)
  • on the other hand, forcing users to change their password regularly leads to weaker passwords, because users tend to choose passwords that are easy to remember, which often ends in _01, _02, _03 passwords.
  A safer solution would be to urge users to pick strong passwords made by a dedicated generator and train them to password managers (like KeePassXC), and let them decide when to change passwords (I don't mean to refrain users from changing their passwords, but —once they have understood the need for truly strong passwords and use them with password managers— changing a strong password for another strong password adds little to the overall system security).
• Patterns suck :
  • qeadzcwrsfxv1331 looks long and solid, but it's actually a keyboard pattern on a QWERTY keyboard. Crackers have methods to test these too.
  • when generating passwords, patterns and rules decrease security :
    • strength comes from randomness (and we —humans— suck at this) : let a computer do the job for you !
    • P@ssw0rd_1234! has uppercase + lowercase + special + digits + is 15 characters long, so it must be very strong, right ?
• Use distinct passwords across sites : in a users database, your password is linked to your email address, and your email address is present in all sites you have an account on. If your password is discovered, it becomes very easy to gain access to your other accounts
• Any string that was once someone's password may be in a cracker's personal dictionary :
  • momof3gr8kids (mom of 3 great kids) was broken thanks to momof3 and gr8kids (which were probably previously broken via brute-force).
  • So if your password is
    • the concatenation of strings anybody already used as password
    • any string someone might already have thought of as for a "strong" password (even LAwL!:'"Yo7vY2u=Z#ti4VbcK!5>@f!%W;_- would be weak if widely used)
    it's actually weak. Use a generator.
• Making a password out of 7 random words from a "large" list is considered safe.
• Know when a password is "strong enough" : if it takes forever to brute-force it, doubling its length won't require crackers to run their programs until infinity twice to break it : if somebody wants your password hard enough, they'll find a way

The F.U.D. strategy

• Fear : Gather facts / data / statistics so that the target people sees everybody has something to fear, and that "feeling safe" is different from "being safe"
• Uncertainty : Everything has an uncertain side, just because some facts are unknown to everybody, things don't always behave as expected, and the future is unknown until it happens.
• Doubt : Make the target people doubt about what they know (or think they know). Show them their understanding has (or can have) failures.

Changing one's mind

Statistically, when people make a decision which appears to be wrong afterwards, they often stick to their initial decision anyway. It seems that admitting one's made a mistake is more difficult than persevere in a wrong way. This works even better when occurring in public.

The "Tell a friend" strategy

Good friends are known to share secrets. Let's imagine you have something "hard" to say to Alice but you have no idea on how to let her know without hurting her feelings. Well, just tell it to Bob, who is Alice's best friend. Say it as a secret, and ask Bob never to repeat this to anyone. Bob's first thought will be to let Alice know.

The "SEsGI" strategy

The Someone Else's Good Idea. Some ideas can not be "good" ideas just because they are yours. (But your boss's ideas are always good). If you want an idea to be thought as "good", just go to people saying "Hey, I just remember the idea you told me about few days ago, and (blah blah blah...)". Very few people are honest enough to answer "Sorry, this wasn't my idea".