Enigmatic Code

Programming Enigma Puzzles

Notes

15 Comments

Here are some notes that I hope will prove useful for contributors to the site:

Including code in WordPress
Including maths in WordPress
Python 2 or Python 3?
Python style
Timing Python programs
Rating Enigma puzzles

Including code in WordPress

The best solution seems to be to use [code language="python"]...[/code] tags in your comment (with the language adjusted accordingly). See this support page for details (or this one).

If you are posting a large section of code (longer than 80 lines or so), please also use the collapse="true" parameter.

Prior to finding out about this I suggested you could just use <code>...</code> tags, but I used to use HTML <pre>...</pre> tags and filter the code with the following Python 3 program (which I call pre-cat) to escape HTML entities (although this makes it difficult to easily switch to using [code] tags, as the HTML entities get escaped twice):

# process a text file suitable for inclusion in HTML tags

import fileinput
import html

for line in fileinput.input():
  print(html.escape(line, quote=False), end='')

Including maths in WordPress

If you know the LaTex document markup language the good news is you can include LaTex directly in WordPress, by using the $latex ...$ construct.

So for example by entering:

$latex d = \frac{ 25 }{ sin(\theta) } - \frac{ 24 }{ tan(\theta) }&s=2$

I get the following expression:

$d = \frac{ 25 }{ sin(\theta) } - \frac{ 24 }{ tan(\theta) }$

(as used in the solution to Enigma 1728).

Python 2 or Python 3?

I try to write code that is portable between Python 2 and Python 3 (at the time of writing I am using Python 2.7.11 and Python 3.5.1). I use the new-style print function and string formatting, which means if you’re executing the code under Python 2 you’ll need to:

from __future__ import print_function

(although now I just tend to use the printf() function from the enigma.py library, which works in both Python 2 and Python 3 and makes interpolating variables much easier than the standard print statement/function).

Python 3.3 introduced the yield from ... construct, which makes writing recursive generators neater. I occasionally use this, and where I do the program will only run in Python 3.3 or later.

If you want to run code that uses yield from ... under Python 2 try replacing:

yield from EXPRESSION

with:

for x in EXPRESSION: yield x

Python style

I tend to use 2-space indents for whatever language I’m using, so naturally I also do this for Python, which will no doubt annoy some of the Python purists, but it does let code fit better in WordPress comments, and it looks better to me than 4-space indents.

If it worries you unduly run it through:

unexpand -t 2 | expand -t 4

Timing Python programs

I use my shell’s (Z-shell) built-in time function, and report the total time (in this case 0.125s or 125ms):

% time python enigma1609.py
5468 [4, 4, 5, 4, 4, 4, 5] 5
python enigma1609.py 0.11s user 0.01s system 97% cpu 0.125 total

The time reported includes all phases of execution from Python start-up, compilation and execution, but you can also use the cProfile module to get an idea of the number of function calls involved (although this will vary between different Python versions) and the internal execution time of the code:

% python -mcProfile -scumulative enigma1609.py
5468 [4, 4, 5, 4, 4, 4, 5] 5
         120855 function calls (120695 primitive calls) in 0.114 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.095    0.095    0.114    0.114 enigma1609.py:4()
   103218    0.008    0.000    0.008    0.000 {len}
     2800    0.004    0.000    0.006    0.000 {filter}
    11088    0.002    0.000    0.002    0.000 enigma1609.py:26()
       93    0.000    0.000    0.002    0.000 enigma.py:57(factor)
   253/93    0.001    0.000    0.002    0.000 enigma.py:65(_factor)
       78    0.000    0.000    0.002    0.000 enigma.py:53(prime)
      192    0.001    0.000    0.001    0.000 {range}
        1    0.001    0.001    0.001    0.001 __future__.py:48()
     2645    0.001    0.000    0.001    0.000 {divmod}
        1    0.000    0.000    0.000    0.000 enigma.py:19()
      280    0.000    0.000    0.000    0.000 {method 'append' of 'list' objects}
      191    0.000    0.000    0.000    0.000 {math.sqrt}
        1    0.000    0.000    0.000    0.000 {print}
        2    0.000    0.000    0.000    0.000 {map}
        7    0.000    0.000    0.000    0.000 __future__.py:75(__init__)
        2    0.000    0.000    0.000    0.000 {method 'count' of 'list' objects}
        1    0.000    0.000    0.000    0.000 __future__.py:74(_Feature)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

For more flexible timing options in Python you can use the Timer class in my enigma.py library.

Rating Enigma puzzles

I have enabled ratings for the puzzles on the site, so you can express an opinion as to what you think of a puzzle.

The ratings go from 1 to 5 stars. I rate puzzles according to how much fun it is to program a solution. So a rating of 1 means the puzzle is trivial to program a solution for (e.g. Enigma 1494). A rating of 3 is an averagely challenging Enigma puzzle. And a rating of 5 is a fun programming challenge.

The list of Challenging Enigmas are all puzzles I have awarded 4 and 5 stars to.

15 responses to “Notes”

Brian Gladman 1 July 2013 at 8:53 am
In view of the recent problems with the measurement of the running times of Python code on different machines, it might be useful to have a timing capability in enigma.py. Here is one possible version that provides for overall timing and subroutine timing using a decorator.
```
from __future__ import print_function
from functools import wraps

try:
  from time import perf_counter as _timer
except ImportError:
  from time import time as _timer

class Timer(object) :
  def __enter__(self): self.start = _timer()
  def __exit__(self, *args):
    t = _timer() - self.start
    if t < 1.0:
      print(' ({:0.3f} ms)'.format(1000 * t))
    else:
      print(' ({:0.3f} seconds)'.format(t))

def timing(f):
  @wraps(f)
  def _timing(*arg):
    t = _timer()
    r = f(*arg)
    t = _timer() - t
    print('{:s} took {:0.3f} ms'.format(f.__name__, 1000.0 * t))
    return r
  return _timing
```
Reply
- Brian Gladman 1 July 2013 at 9:25 am
  Here is an example of use:
```
from timer import Timer, timing
from enigma import Primes

@timing
def get_primes(n):
  return Primes(n).list()

with Timer():
  get_primes(10000000)
```
  Reply
  - Jim Randell 16 August 2013 at 2:00 pm
    I’ve folded some of these ideas into a timing module I wrote a while back. If you want to try it out you can download it at this link [ http://www.magwag.plus.com/jim/timing.py ]. There’s some documentation in the file. I haven’t tested it in a MS Windows environment, so it would be good to know if it works there. (I’ve put code in to use time.clock on Windows, as timeit does).
    
    Of course it will report very different timings to the run times I usually report as this is measuring an internal elapsed time to the Python interpreter, rather than an all inclusive elapsed time of the whole thing. For instance, my solution to Enigma 1762 is reported as taking 776µs, rather than 38ms. (Both of which still count as “instantaneous”).
    
    I’ve not included it directly into enigma.py at the moment because I like the fact you can just add import timing to your code and get a pretty good measure of the internal runtime, but I don’t think I want that to happen if you import enigma.py. (I could only enable the default time only if you have some environment variable set, but that seems a bit messy).
    
    Reply
    - Jim Randell 16 August 2013 at 4:44 pm
      Actually I think I’ve persuaded myself that I could put most of this into enigma.py, and have a special Timer object you can import. Then you would be able to wrap the code you wanted to time with calls to [[ timer.start() ]] and [[ timer.stop() ]]. Then at least it would be explicit what you are timing. And the rest of it would still be available for people who want to use the more esoteric functionality.
      
      Update: This is now included in the latest version (2013-08-16) of enigma.py.
      
      Reply
- Jim Randell 7 August 2013 at 11:22 am
  For measuring the timing of snippets of code Python has its own module called [[ timeit ]], which says it “avoids a number of common traps for measuring execution times”, so I’m happy to use it rather than roll my own code (which may well stumble into several of these common traps).
  
  You could use it like this:
```
% python -mtimeit -s'from enigma import Primes' 'sum(Primes(10000000))'
```
  For overall program execution time I do multiple runs of the program and report the shortest elapsed time (as reported by my shell’s time builtin). I did consider having a “Raspberry Pi” metric to allow people to estimate comparable runtimes with minimal outlay, but really the times I report are only “tourist information” to give a rough idea of how much work the program is doing, as accurate timings depend on all aspects of the system the code is executing on.
  
  But generally I tend to consider my programs to fall into roughly the following categories:
  
  Instantaneous: ≤ 100ms
  Fast: ≤ 1s
  OK: ≤ 10s
  Slow: ≤ 100s
  Better than nothing: > 100s
  
  And usually try to aim for a “Fast” solution (although I will prefer readability and correctness of a program over execution speed).
  
  The advantage of using the shell’s timing the program is that it gives me a way of comparing the execution time of programs that use different programming environments. When I started coding Enigma puzzles I would do them in Perl, and I’m able to use the same technique to estimate the run time of those programs (and if I were comparing compiled code I would think it would be only fair to include the compilation time into the overall runtime). Although I am aware it may be trickier to get such timings under MS Windows based systems.
  Reply
  - Brian Gladman 7 August 2013 at 5:07 pm
    I agree that timing can be difficult and can also depend on what the objective of the timing really is. But I have noticed a number of people who are adding timing code to their Python programs in various ways, some of which are highly suspect and others that are just plain wrong. So I am still of the view that it would make sense to offer a common way of doing timing for those who want to add timing capabilities into their code.
    
    Reply
    - Ahmet Saracoğlu 7 August 2013 at 6:13 pm
      Hi to all,
      
      What about the cpu’s speed? That does matter, so I suggest that Jim should measure each one’ code on his machine, that is more true
      
      Reply
      - Brian Gladman 7 August 2013 at 10:33 pm
        Yes that matters but people can simply announce the speed of their machines if this matters. Its not perfect but a common approach would avoid the big errors we have seen recently in timing. Jim can speak for himself but I am doubtful that he would be willing to take on the task you suggest given the limited benefits and the added costs involved.
        
        Reply
        
        Jim Randell 14 August 2013 at 10:22 am
        Actually I don’t mind reporting comparative timings for code posted on the site (if I can get it to run). Just ask when you post the code and I’ll do a comparative timing run on it.
        
        Reply

Frits 11 December 2020 at 10:37 am

My preferred way of running Python programs is with PyPy and enigma.py for timings.
Also preprocessing takes place so debug statements are converted,

The Python program has the following function to make the program also run with CPython:

def pr(*args, **kwargs):
  print(*args, **kwargs)

fi, preprocessing will change all pr(x, y) statements to print("x =", x, "y =", y)

Example:

D:\$FTV\Python>pypy boxclever.py
(2, 3, 4, 6, 2, 1, 3, 3, 3, 6)
[temp.py] elapsed time: 0.0625000s (62.50ms)

pypy.bat

@echo off
if exist %1 (
  copy %* d:\$ftv\python\temp.py > NUL
  c:\pypy\pypy3 d:\$ftv\python\preprocessing.py d:\$ftv\python\temp.py
  c:\pypy\pypy3 tempMain.py
  del temp.py 2>nul
)

tempmain.py

import enigma
enigma.run("temp.py", timed=1)

preprocessing.py

# preprocess the Python program 
import sys
from re import findall
 
# change all lines with <pr(x, y)> to:
# <print("x =", x, "y =", y)>  
#
# and
#
# change all lines with <pl(x, d)> to:     
# <print("--- list x (n entries) ---)"
#  for k in x: print(k)>  
# <print("--- list d (n entries) ---)"
#  for k in d.items(): print(k)>       if d is a dictionary

lines_sep = "\n"

def find_number(text, c):
    return findall(r'%s(\d+)' % c, text)
    
def pr_func(x):
  pos1 = x.find("[")
  res = '"' + x + ' =", ' + x
  if pos1 < 0 or x.count("[") > 1:
    return res
  else:  
    pos2 = x.rfind("]")
    pos3 = x.find(":")
    if pos2 < 0: 
      return res
    
    if pos3 > pos1 and pos3 < pos2:
      return res
      
    return '"' + x[:pos1] + '[" + str(' + x[pos1 + 1:pos2] + ') ' + \
           ' + "' + x[pos2:] + '" + " =", ' + x
  
def pl_func(x, ind, lines=9999):
  # generators don't have a length
  s = '"--- list generator ' + x + ' ---" if type(' + x + \
      ') == gentyp else "--- list ' + x + \
      ' (" + str(len(' + x + ')) + " entries) ---")'
  s += lines_sep + ' '*ind
  
  # print keys and values if items() is not specified for a dictionary
  s += "for kx in [ky for iy, ky in enumerate(" + x + \
       ".items() if isinstance(" + x + ", dict) else " + x + ") if iy < " 
  s += str(lines) + "]: print(kx" 
  return s
  
def preprocess(i, ln, keys, prt):
  
  for key in keys:
    pos = ln.find(key)
    ind = len(ln) - len(ln.lstrip())   # indentation
    if prt: 
      print("key", key, "pos", pos, "ind", ind, "ln", ln[:-2])

    # skip if keyword not found or it occurs in a function definition
    if pos == -1 or ln.find("def ") > -1: continue
    # skip if there is a non-blank for the keyword 
    if pos > 0 and ln[pos - 1] != " ": continue
    # skip if there is a # before the key word
    poscom = ln.find("#")
    if poscom > -1 and poscom < pos: continue

    var = ""
    part1 = ""    
    prPrint = ln[:pos]  + "print("
    if prt: print("prPrint", prPrint)

    lev = 0
    
    numlines = find_number(ln, "lines=")
    # check each character
    for j, x in enumerate(ln[pos + 3:]):
      if x == "#": break
      # don't treat comma's in func params, lists and sets as print separators
      if x in {"(","[", "{"}:
        lev += 1
      if x in {")","]", "}"}:
        lev -= 1

      if x == "," and lev == 0:  # this comma is used to separate print vars
        var = var.strip()
       
        if var[0] == '"' or var[0] == "'":
          new = var
        else:
          if key == "pr(":
            new = pr_func(var)
          if key == "pl(":
            if numlines != []:
              new = pl_func(var, ind, lines=int(numlines[0]))
            else:
              new = pl_func(var, ind)
            

        part1 += prPrint + new
        if key == "pr(":
          part1 += ', '  # prepare for next variable
        if key == "pl(":
          if numlines == []:
            part1 += ")" + lines_sep + " "*ind + "print("
          
        prPrint = ""             # only use print( once
        lines[i] = part1 + lines[i][pos + 4 + j:] 
        var = ""                 # variable has been handled

      if lev > 0 or (lev == 0 and x not in ",\n"):  
        var += x
      if lev == -1 and x == ")":
        break

    if lev == -1:                # print closing parenthesis found
      var = var.strip()
      if var[0] == '"' or var[0] == "'" or var[:3] == "end":
        new = var
      else:
        if key == "pr(":
          new = pr_func(var)
        if key == "pl(":
          new = "" if var[:6] == "lines=" else pl_func(var, ind)
         
      if part1.strip() == "":
        part1 = prPrint
      
      lines[i] = part1 + new + ln[pos + 3 + j:] 
    

if __name__=="__main__":

  outputfile = inputfile = sys.argv[1]
  prt = sys.argv[2] if len(sys.argv) > 2 else 0
  
  with open(inputfile) as ifile:
    lines = ifile.readlines()
  
  keys = ["pr(", "pl("]
  
  lines[0] = 'gentyp = type(1 for i in "")\n' + lines[0]  # check if generator

  # process all lines of input file
  for i, x in enumerate(lines): 
    preprocess(i, x, keys, prt)          # change <pr(x)> to <print("x =", x)>  
  
  if prt: 
    for x in lines: print(x, end="")
    #for x in olines: print(x, end="")
    print()
   
  # write to file 
  with open(outputfile, 'w') as ofile:
    ofile.writelines(lines)

@Jim, do you know an easier way to parse a string, like pr(a, sum([a, b])), into arguments a and sum([a, b])?

Jim Randell 11 December 2020 at 12:45 pm
@Frits: Did you know about the [[ f"{<expr>=}" ]] added to f-strings recently?
```
% python3.9
Python 3.9.1 (default, Dec  8 2020, 11:42:15) 

>>> (x, y) = (37, 42)
>>> f"{x=} {y=}"
'x=37 y=42'

>>> s = (1, 2, 3, 4)
>>> f"{s[:2]=}"
's[:2]=(1, 2)'
```
This was added in Python 3.8 I believe, so will require a recent version of Python (or you will get an error).

(This will also work with printf() and sprintf() from the enigma.py library, as they use f"..." strings on more recent Python versions. Of course it will only work with versions of Python that support the new syntax).
Reply
- Frits 11 December 2020 at 10:11 pm
  @Jim, Thanks. I didn’t know about this.
  
  I prefer to write “pr(x, y)”.
  
  Unfortunately the variable name still gets lost when passed in a function so preprocessing is needed when I want to use “pr”.
  
  line 13 and 14 have been corrected in preprocessing.py (for indentation errors):
```
part1 = ""    
prPrint = ln[:ind]  + "print("
```
  Reply

Frits 9 July 2022 at 12:35 pm

Instead of preprocessing you can also include this function for quick debugging.

   
from inspect import currentframe, getouterframes

# (a, b, c) = (1, 4, 5) and seq = [a, b, c]
# pr("test", a, b, c, seq, seq[a]) results in output: 
# test a = 1 b = 4 c = 5 seq = [1, 4, 5] seq[1] = 4
def pr(*args):
    funcname = pr.__name__
    # return the string within the code that called funcname
    parm_str = getouterframes(currentframe())[1][4][0]
    # get rid of funcname( and \n
    parm_str = parm_str[parm_str.index(funcname) + len(funcname) + 1:-2]
    var_names, var_values = list(split_str(parm_str)), args
    
    # substitute indices, eg if i = 2 then replace g[i] by g[2]
    # internally use variable names longer than 2 chars
    for iii in range(len(var_names)):  
      xxx = var_names[iii]
      pos1 = xxx.find("[")
      if pos1 < 0: continue
      pos2 = pos1 + xxx[pos1:].find("]")
      if pos2 < 0: continue
      vvv = xxx[pos1 + 1:pos2]
      # skip substitution for slicing or if index name is longer than 2 chars
      if ":" in vvv or len(vvv) > 2: continue
      
      if vvv in globals(): 
        var_names[iii] = xxx[:pos1 + 1] + str(eval(vvv)) + xxx[pos2:] 
    
    res = "".join(str(vs) + " "
             if str(k)[-1] in {"'", '"'} 
             else str(k).lstrip() + " = " + str(vs) + " "
             for k, vs in zip(var_names, var_values))            
    print(res)

ilya 31 July 2022 at 3:10 pm
Jim, what am I doing wrong? The code-block thing simply doesn’t work for me when I’m commenting on your site (it does work when I comment in my own wordpress sandbox).
Here is what I get:
```
void foo(int bar) {}
```
It does work when I’m trying to post comment in my own sandbox wordpress site.
Reply
- ilya 31 July 2022 at 3:10 pm
  Hmm.. so it does work. Weird.
  
  Reply

	Jim Randell on Enigma 1425: Spell check
	Jim Randell on Tantalizer 13: Balls
	Jim Randell on Enigma 937: Progressive f…
	Frits on Enigma 937: Progressive f…
	Jim Randell on Enigma 838: Time for inso…
	ilya on Headscratcher #224: Russian…
	Jim Randell on Headscratcher #224: Russian…
	ilya on Headscratcher #224: Russian…

Enigmatic Code

Notes

Including code in WordPress

Including maths in WordPress

Python 2 or Python 3?

Python style

Timing Python programs

Rating Enigma puzzles

15 responses to “Notes”

Leave a Comment Cancel reply

Random Post

Recent Posts

Recent Comments

Archives

Categories

Follow via email

Site Stats

Meta

Enigmatic Code

Notes

Including code in WordPress

Including maths in WordPress

Python 2 or Python 3?

Python style

Timing Python programs

Rating Enigma puzzles

Share this:

Like this:

15 responses to “Notes”

Leave a Comment Cancel reply

Random Post

Recent Posts

Recent Comments

Archives

Categories

Follow via email

Site Stats

Meta