url blocking again!

An area specifically for port forwarding, firewalls and other (on-line) security related issues.
Post Reply
duke
Regular
Regular
Posts: 61
Joined: Mon Nov 26, 2012 7:03 pm

url blocking again!

Post by duke » Tue Feb 19, 2013 5:39 pm

:mrgreen:
sorry it gets asked in here alot but

is there a way to add loads of website urls in a list and block them
or if not urls then ipaddress range blocks

and not just plain urls also regular expression pattern matching blocks

i've read faq and features.html pages tinyproxy is nice
http://wiki.openwrt.org/doku.php?id=old ... .tinyproxy
in open wrt is it available to routertech FW


i'm going to configure dnsmasq
using this guide
http://wiki.openwrt.org/doc/howto/dhcp.dnsmasq
and stick my list in etc/hosts


since the list of sites to block is mosly url but
i would like to know if in any way we can do something like simple regex
*facebook.com/ajax/chat* <- this blocks facebook chat without blocking facebook (* means anything and everything preceding or following in url but they must contain "facebook.com/ajax/chat" blocking the facebookchat via ipadress is not feasible )

what i want is something to make it sticky between reboots ,the original firmware does it is there any way in routertech firmware
and without downloading a adblock file (i dont want that feature as it will fight for system resources if/when my ip block list grows )

and minix partition might get corrupted ? if there is a reboot without unmounting (eg. in a power loss )
is there no way to store it in rom and load automatically
User avatar
thechief
RouterTech Team
RouterTech Team
Posts: 12066
Joined: Wed Feb 01, 2006 10:22 pm
Location: England, the Centre of Africa
Contact:

Re: url blocking again!

Post by thechief » Tue Feb 19, 2013 6:20 pm

The Chief: :afro: Be sure to read the Firmware FAQ and do a Forum Search before posting!
No support via PM. Ask all questions on the open forum.
duke
Regular
Regular
Posts: 61
Joined: Mon Nov 26, 2012 7:03 pm

Re: url blocking again!

Post by duke » Tue Feb 19, 2013 6:42 pm


my list is a long one is there a way to pass a file as a parameter to the command (i dont think so)

also
setenv autoexec.sh "block.sh facebook.com && block.sh twitter.com && block.sh orkut.com"
setenv autoexec1.sh "block.sh blah_blah && block.sh bloo_bloo && block.sh blee_blee"
is there a limit to number of parameters using this(&&) ?

i prefer the dnsmasq way of loading the file from etc/hosts is there no way of making changes to hosts file permanent? (so i just make the initial configuration and the program automatically picks up sites to block from a saved file on router, and any changes i make to the file gets reflected at next reboot)


but also i want dnsmasq to be active without using adblock features or if using adblock persistance (no downloading adblock file at each restart{i get a little paranoid])

i know its available for download but according to openwrt docs i need other packages too
in http://wiki.openwrt.org/doku.php?id=old ... .tinyproxy
i need
You have to install the following packages on the router:

ipkg install kmod-ipt-nat
ipkg install iptables-mod-nat
ipkg install tinyproxy
i know routertech had tinyproxy before
if i use tinyproxy is there a guide for routertech how do i install/configure ?
User avatar
thechief
RouterTech Team
RouterTech Team
Posts: 12066
Joined: Wed Feb 01, 2006 10:22 pm
Location: England, the Centre of Africa
Contact:

Re: url blocking again!

Post by thechief » Tue Feb 19, 2013 9:00 pm

duke wrote:my list is a long one is there a way to pass a file as a parameter to the command (i dont think so)
No.
duke wrote:
setenv autoexec.sh "block.sh facebook.com && block.sh twitter.com && block.sh orkut.com"
setenv autoexec1.sh "block.sh blah_blah && block.sh bloo_bloo && block.sh blee_blee"
is there a limit to number of parameters using this(&&) ?
The length of the command line?
duke wrote:i prefer the dnsmasq way of loading the file from etc/hosts is there no way of making changes to hosts file permanent?
No. The router has a readonly file system. The only persistent things are the firmware itself, the config block, the environment block, and whatever you store in a minix partition.
duke wrote:but also i want dnsmasq to be active without using adblock features or if using adblock persistance (no downloading adblock file at each restart{i get a little paranoid])
a. Create your block file in dnsqmasq format
b. copy it to /var/tmp/dns.d/ on the router
c. run "killall dproxy"

That is all you need to do.
duke wrote:i know routertech had tinyproxy before
if i use tinyproxy is there a guide for routertech how do i install/configure ?
The package on our repository contains the necessary scripts that were used when tinyproxy was part of the firmware. Read them and amend them as required.
The Chief: :afro: Be sure to read the Firmware FAQ and do a Forum Search before posting!
No support via PM. Ask all questions on the open forum.
duke
Regular
Regular
Posts: 61
Joined: Mon Nov 26, 2012 7:03 pm

Re: url blocking again!

Post by duke » Tue Feb 19, 2013 9:35 pm

a
. Create your block file in dnsqmasq format
b. copy it to /var/tmp/dns.d/ on the router
c. run "killall dproxy"
i need help with this

i created the file uploaded it to the router

but how do i make it stick between reboots :curse:

when i do
setenv autoexec.sh block.sh xyz.com

so that block.sh gets executed at each reboot
the .conf file is deleted and a new one with just xyz.com in dnsmasq format is visible
User avatar
thechief
RouterTech Team
RouterTech Team
Posts: 12066
Joined: Wed Feb 01, 2006 10:22 pm
Location: England, the Centre of Africa
Contact:

Re: url blocking again!

Post by thechief » Tue Feb 19, 2013 10:11 pm

Your best bet is to create a minix partition, and store your block file there. Then you need to create a startup.sh on the minix partition, with all the stuff you want to do - e.g.,

Code: Select all

#!/bin/sh
echo "Starting to run /nvram/startup.sh" | logger
cp /nvram/myblockfile.conf /var/tmp/dns.d/
killall dproxy
block.sh xyz.com
blah blah
blooh blooh
goody goody
whatever
echo "Finished running /nvram/startup.sh" | logger
The piping of the "echo" command to "logger" simply adds the message to the system log. This enables you to verify (by looking at the system log) that the /nvram/startup.sh script is actually executed on bootup.
The Chief: :afro: Be sure to read the Firmware FAQ and do a Forum Search before posting!
No support via PM. Ask all questions on the open forum.
duke
Regular
Regular
Posts: 61
Joined: Mon Nov 26, 2012 7:03 pm

Re: url blocking again!

Post by duke » Sat Feb 23, 2013 3:26 am

ok
thanks for that chief

i have some questions about the minix partition
1 if its not unmounted properly does the data in it gets lost / will it not remount at next reboot ?
2 is there a time limit between mounting and unmounting the partition /(a safe limit)
what i want to do is mount the partition run scripts(..save data..whatever) and then quickly unmount it and do this each time i want the data in partition to be accessed so the partition is not corrupted

3. in future if my need outgrows the space or i have a 4mb router , is there an easy way to add sd card support using gpio mod ?(i know usb ports are cheaper now, but ar7 is on its way out and the platform is stable [aside from heating cpu's and bad capacitors {which are are known unknowns :lol: }])

4 what i am thinking is a script that does stuff then unmounts the minix partition (for example your startup.sh) /
is there a way to do that inside a script (can i call a script from another script)/ what command should i add in startup.sh
User avatar
thechief
RouterTech Team
RouterTech Team
Posts: 12066
Joined: Wed Feb 01, 2006 10:22 pm
Location: England, the Centre of Africa
Contact:

Re: url blocking again!

Post by thechief » Sat Feb 23, 2013 10:55 am

duke wrote: i have some questions about the minix partition
1 if its not unmounted properly does the data in it gets lost / will it not remount at next reboot ?
You might have data loss (only of new data that has not been flushed). You can prevent such data loss by running "flush" after writing any data to the minix partition.
duke wrote:2 is there a time limit between mounting and unmounting the partition /(a safe limit)
Who knows? It will behave just like any old linux system.
duke wrote: what i want to do is mount the partition run scripts(..save data..whatever) and then quickly unmount it and do this each time i want the data in partition to be accessed so the partition is not corrupted
Why would the data get corrupted?
duke wrote:3. in future if my need outgrows the space or i have a 4mb router , is there an easy way to add sd card support using gpio mod ?
No.
duke wrote:4 what i am thinking is a script that does stuff then unmounts the minix partition (for example your startup.sh)
Why would you want to unmount it? Leaving it alone would do it no harm.
duke wrote:is there a way to do that inside a script (can i call a script from another script)/ what command should i add in startup.sh
Same way as you run any old command from a script. You might however discover that you are unable to unmount the filesystem while still running a script that resides there.
The Chief: :afro: Be sure to read the Firmware FAQ and do a Forum Search before posting!
No support via PM. Ask all questions on the open forum.
mstombs
RouterTech Team
RouterTech Team
Posts: 3753
Joined: Wed Jan 10, 2007 11:54 pm

Re: url blocking again!

Post by mstombs » Sat Feb 23, 2013 12:40 pm

thechief wrote:
duke wrote: i have some questions about the minix partition
1 if its not unmounted properly does the data in it gets lost / will it not remount at next reboot ?
You might have data loss (only of new data that has not been flushed). You can prevent such data loss by running "flush" after writing any data to the minix partition.
1. You can have data loss BTDTGTTS, because the OS does buffer and delay writes (jffs is more advanced filesystem format that does this better for flash - but is bigger and needs more flash blocks to work with than minix).

Note that 'flush' is a RouterTech firmware script, especially designed to overcome this - the Linux command 'sync' (potientially provided by BusyBox) didn't do the job for us.

Have you considered cifs from a remote PC/NAS?
duke
Regular
Regular
Posts: 61
Joined: Mon Nov 26, 2012 7:03 pm

Re: url blocking again!

Post by duke » Thu Mar 21, 2013 7:24 pm

duke wrote:
what i want to do is mount the partition run scripts(..save data..whatever) and then quickly unmount it and do this each time i want the data in partition to be accessed so the partition is not corrupted
Why would the data get corrupted?
i am expecting frequent power loss/failures at the location where the router is going to be at. (so if it isint unmounted properly there would be no harm to minix)

the whole reason for creating the minix is for blocking custom url's / web links.so there wont be a frequent writing to the partition just the reading variables/running scripts at boot time to inject the urls and setup the filter ! thats it

:roll:
the specs are
RAM 16.0 MB
Flash (ROM) 4.0 MB

mtd0 0x9009E000,0x90400000
mtd1 0x90020090,0x9009E000
MTD
dev: size erasesize name
mtd0: 00362000 00010000 "mtd0"
mtd1: 0007df70 00010000 "mtd1"
mtd2: 00010000 00002000 "mtd2"
mtd3: 00010000 00010000 "mtd3"
mtd4: 003e0000 00010000 "mtd4"

System environment (/proc/ticfg/env)
BUILD_OPS 0x301
bootloaderVersion 1.4.0.4
ProductID AR7WRD
HWRevision Unknown
SerialNumber none
IPA 192.168.1.1
MAC_PORT 0
MEMSZ 0x01000000
FLASHSZ 0x00400000
MODETTY0 9600,n,8,1,hw
MODETTY1 9600,n,8,1,hw
CPUFREQ 211968000
MIPSFREQ 211968000
SYSFREQ 105984000
BOOTCFG m:f:"mtd1"
PROMPT (psbl)
mtd2 0x90000000,0x90010000
mtd3 0x90010000,0x90020000
mtd4 0x90020000,0x90400000
StaticBuffer 384
IPA_SVR 192.168.1.2
wlan_id 1x50
vlynq_polarity low
DSL_BIT_TMODE 1
sar_ipacemax 6
DSL_UPG_DONE 1
vcc_encaps0 0.0
vcc_encaps1 0.0
vcc_encaps2 0.0
vcc_encaps3 0.0
vcc_encaps4 0.0
vcc_encaps5 0.0
vcc_encaps6 0.0
vcc_encaps7 0.0
usb_vid 0x0451
usb_pid 0x6060
HWA_RNDIS 00:E0:A6:66:41:EB
HWA_HRNDIS 00:E0:A6:66:41:E1
autopvc_enable 1
pppoe_relay_enabled 1

unrelated question
does anyone know what static buffers are what are they used for ?? mine are "StaticBuffer 384" are they like swap space for connections?
edit almost forgot
what parameter should i pass to auto minix.sh? what is a safe size for the partition??
User avatar
thechief
RouterTech Team
RouterTech Team
Posts: 12066
Joined: Wed Feb 01, 2006 10:22 pm
Location: England, the Centre of Africa
Contact:

Re: url blocking again!

Post by thechief » Fri Mar 22, 2013 11:10 pm

duke wrote:does anyone know what static buffers are what are they used for ??
Dunno.
duke wrote:what parameter should i pass to auto minix.sh? what is a safe size for the partition??
You don't run auto_minix.sh at all. You should run makemtd.sh. 384 is a sensible maximum for a wireless firmware.
The Chief: :afro: Be sure to read the Firmware FAQ and do a Forum Search before posting!
No support via PM. Ask all questions on the open forum.
duke
Regular
Regular
Posts: 61
Joined: Mon Nov 26, 2012 7:03 pm

Re: url blocking again!

Post by duke » Sun Mar 31, 2013 1:21 am

:lovesight:

proud to say that it worked
, everything

i actually had tinyproxy downloaded from the binaries posted in the forum
but i dont need it it seems

created minix partition

so just winscp into the folder
created startup.sh { <-- the block file name must be " block_user.conf "}

here is my startup.sh file that is placed in nvram folder along with block_user.conf file

Code: Select all

#!/bin/sh
echo "Starting to run /nvram/startup.sh" | logger
cp /nvram/block_user.conf /var/tmp/dns.d/
killall dproxy
block.sh xyz.com
echo "Finished running /nvram/startup.sh" | logger
addresses in block_user.conf should be 1 per line
and in " address=/example.dev/127.0.0.1 " format

also it seems dnsmasq is able to take wildcards as parameter
example: *facebook.com/ajax/chat* is modified as facebook.com/ajax/chat
and apparently works
:lol:
User avatar
thechief
RouterTech Team
RouterTech Team
Posts: 12066
Joined: Wed Feb 01, 2006 10:22 pm
Location: England, the Centre of Africa
Contact:

Re: url blocking again!

Post by thechief » Sun Mar 31, 2013 4:36 pm

:)
The Chief: :afro: Be sure to read the Firmware FAQ and do a Forum Search before posting!
No support via PM. Ask all questions on the open forum.
Post Reply