Cron Job -> external API request -> write XML

RistoKaalma · December 15, 2019, 1:50am

I’m trying to set up a cron job that runs every day, makes an API request and writes a XML file to server that can be later used to import Woo products to DB using WP ALL IMPORT. I’ve bee trough quite a few topics here about cron jobs but I’m lost now.

/trellis/roles/wosrpress-setup/tastks/main.yml

- name: Cron Job Name
  cron:
    name: "{{ item.key }} Cron Job Name"
    hour: "{{ item.value.cron_trigger.hour | default('*') }}"
    minute: "{{ item.value.cron_trigger.minute | default('*') }}"
    user: "{{ web_user }}"
    job: "cd {{ www_root }}/{{ item.key }}/{{ item.value.current_path | default('current') }} && {{ item.value.cron_trigger.job }} > /dev/null 2>&1"
    cron_file: "api-cron-job-trigger"
    state: "{{ (cron_enabled and not item.value.multisite.enabled) | ternary('present', 'absent') }}"
  with_dict: "{{ wordpress_sites }}"

/trellis/group_vars/development/wordpress_sites.yml

wordpress_sites:
  clientsite.com:
    site_hosts:
      - canonical: clientsite.test
        redirects:
          - www.clientsite.test
    local_path: ../site # path targeting local Bedrock site directory (relative to Ansible root)
    admin_email: email@domain.com
    multisite:
      enabled: false
    ssl:
      enabled: false
      provider: self-signed
    cache:
      enabled: false
    cron_trigger:
      job: "wget -q -O <what-do-i-put-here-?>/api-folder/get_products.php"
      hour: "9-17"
      minute: "0"

Question 1: How should I target the PHP file on job: line?

The PHP file get_products.php containing functions to make the API call is located in /site/api-folder/

Question 2: Which location would be good to locate my API related folder in trellis/bedrock structure?
Question 3: How can I access .evn values from /trellis/group_vars/all/vault.yml (api_user: “user”, api_pass: “pass”, wp_rest_api_cs: “cd_1231234”, wp_rest_api_ck: “ck_312903”, etc) inside the /site/api-folder/get_products.php ?

My end goal is to write the XML file inside /site/api-folder/product-import/and later on use another cron job to import the products to Woo (http://www.wpallimport.com/documentation/recurring/cron/).

strarsis · December 15, 2019, 3:21pm

My two cents: By default Trellis uses a real system cron job that by default runs all 15 minutes which triggers the WordPress internal “cron” job system to make it check for due jobs to be executed.
So when you set up an internal WordPress job, the system cron job should trigger it, in best case immediately when it is due, in the worst case, 15 minutes too late.

So you may want to reduce the time between system cron job triggers, e.g. to 5 minutes. I shouldn’t be computationally expensive as it is a “lightweight” CLI request. The cron job interval I am currently using for publishing scheduled posts (advent calendar content) is 5 minutes, if there are any people that wait for the advent calendar to open at 00:00 local time, would in best case see the door open at 00:00:01, when the system cron job interval aligns correctly, and in the worst case 00:05 or something when it totally misaligned.

You may also be interested in the WP Pipes plugin, I use it for automatically importing RSS feeds, though honestly, I have been unable yet to really set up its cron job part correctly…

RistoKaalma · December 18, 2019, 3:47pm

Thank’s strarsis, for the alternative way. I’ll most definately look into it as well.

I found a trellis-way to accomplish this. Not sure if that’s the most appropriate way to do it but I moved the directory containing API related stuff to /site/web/app/api-folder/ ingluding the get_products.php file containing API functions.

I then modified my cron job in /trellis/roles/wosrpress-setup/tasks/main.yml to navigate to /site/web/app/api-folder/ and get the job value {{ item.value.cron_trigger.job }} from /trellis/group_vars/development/wordpress_sites.yml like so:

/trellis/roles/wosrpress-setup/tastks/main.yml

- name: Cron Job Name
  cron:
    name: "{{ item.key }} Cron Job Name"
    hour: "{{ item.value.cron_trigger.hour | default('*') }}"
    minute: "{{ item.value.cron_trigger.minute | default('*') }}"
    user: "{{ web_user }}"
    job: "cd {{ www_root }}/{{ item.key }}/{{ item.value.current_path | default('current') }}/web/app/api-folder && {{ item.value.cron_trigger.job }} > /dev/null 2>&1"
    cron_file: "api-cron-job-trigger"
    state: "{{ (cron_enabled and not item.value.multisite.enabled) | ternary('present', 'absent') }}"
  with_dict: "{{ wordpress_sites }}"

Changed the job: value in wordpress_sites.yml to run the php script quietly php -q get_products.php" like so:

/trellis/group_vars/development/wordpress_sites.yml

wordpress_sites:
  clientsite.com:
    site_hosts:
      - canonical: clientsite.test
        redirects:
          - www.clientsite.test
    local_path: ../site # path targeting local Bedrock site directory (relative to Ansible root)
    admin_email: email@domain.com
    multisite:
      enabled: false
    ssl:
      enabled: false
      provider: self-signed
    cache:
      enabled: false
    cron_trigger:
      job: "php -q get_products.php""
      hour: "9-17"
      minute: "0"

Note: If anyone sees a security issue here please let me know. I’d appreciate some useful tips to tidy things up.

Next step would be to add sleep(20); to get_products.php script to wait for 20 seconds after API request has finished and XML has been saved and then call wp-load.php to initiate the WP ALL IMPORT script to update products to Woo.

strarsis · December 18, 2019, 5:38pm

Can’t the get_products.php script invoke the WP All Import script by itself when API request finished?
Waiting some time can introduce race conditions.

swalkinshaw · December 22, 2019, 1:56am

I don’t really have an opinion beyond the cron itself. But your solution for setting up the cron job in Trellis and putting the script in /site/web/app/api-folder/ seems completely reasonable and probably what I’d do

RistoKaalma · December 22, 2019, 2:37pm

Thanks! However, is it possible to access trellis .env values (API credentials, WP REST APi keys, etc) inside /site/web/app/api-folder/get_products.php ? Or would it make more sense to create custom fields in options table & store/query the credentials from WP DB?

TangRufus · December 22, 2019, 5:38pm

Make a WP REST endpoint, then all wp functions, plugins functions, constants (.env + application.php) will be available.

Though I think a wp cli command is a better choice

RistoKaalma · December 22, 2019, 10:19pm

Could you elaborate a bit more? I don’t quite follow.

swalkinshaw · December 26, 2019, 8:32pm

Think he meant a WP-CLI command. But his main point is if you make a solution that “uses” WP (meaning it loads the WordPress environment/code) then all the normal credentials/db access will be taken care of automatically.

If you really wanted to make .env values available in a non-WP context/script, you have two main options:

load the .env file like we do in Bedrock
use a template file in Trellis to template out the values into your script file

system · January 26, 2020, 1:51am

This topic was automatically closed after 42 days. New replies are no longer allowed.