Need help with .htaccess and robots files. Kind of urgent.

tonoshi · 08-23-2010 03:00 AM

Im really struggling with my .htaccess and robots settings for my forum.

I have lost all my listings in google and have 2 pages listed out of 300+

I have been tweeking things in the 2 files for various things and now im lost.

Would someone be able to review the files if i posted them? Please, I really need some help. It is for a vb forum btw.

This is what changes i have made from lastnight to try and get this working

.htaccess

Code:

Order Allow,Deny
Deny from 178.162.161.42
# Deny from 212.113.32.      # Ukraine
# Deny from 212.113.33.      # Ukraine
# Deny from 212.113.34.      # Ukraine
# Deny from 212.113.35.      # Ukraine
# Deny from 212.113.36.      # Ukraine
# Deny from 123.152.         # China Unicom Zhejiang province network
# Deny from 123.153.         # China Unicom Zhejiang province network
# Deny from 123.154.         # China Unicom Zhejiang province network
# Deny from 123.155.         # China Unicom Zhejiang province network
# Deny from 123.156.         # China Unicom Zhejiang province network
# Deny from 123.157.         # China Unicom Zhejiang province network
# Deny from 123.158.         # China Unicom Zhejiang province network
# Deny from 123.159.         # China Unicom Zhejiang province network
# Deny from 22.224.          # USA
# Deny from 81.92.151.       # Czech Republic
# Deny from 115.             # China
# Deny from 122.224.135.     # China
# Deny from 218.61.          # China
# Deny from 218.60.          # China
# Deny from 114.           # China
# Deny from 30.58.205.       # Spain Madrid
# Deny from 217.65.          # Hungury
# Deny from 200.75.          # America and Africa
# Deny from 200.161.         # America and Africa
# Deny from 121.176.         # Korea Republic
# Deny from 78.94.           # Germany
# Deny from 85.62.           # Spain 
# Deny from 188.228.         # Denmark Skanderborg - 188.228.33.237
# Deny from 173.236.         # United States Chicago Singlehop Inc 
Allow from all

# ErrorDocument 403 http://errors.redirectme.net/error403.htm
# ErrorDocument 400 http://errors.redirectme.net/error403.htm

DirectoryIndex index.php

RewriteEngine on
# RewriteRule ^index.php /$1 [R=301,L]
# RewriteRule ^index.html /$1 [R=301,L]
RewriteRule ^forum.php /$1 [R=301,L]

RewriteBase /
RewriteCond %{REQUEST_FILENAME} -s [OR]
RewriteCond %{REQUEST_FILENAME} -l [OR]
RewriteCond %{REQUEST_FILENAME} -d

RewriteRule ^.*$ - [NC,L]
RewriteRule ^threads/.* showthread.php [QSA]
RewriteRule ^forums/.* forumdisplay.php [QSA]
RewriteRule ^members/.* member.php [QSA]
RewriteRule ^blogs/.* blog.php [QSA]
ReWriteRule ^entries/.* entry.php [QSA]
RewriteRule ^list/([^/]*/)([0-9]+) list.php?r=$1$2&%{QUERY_STRING}
RewriteRule ^content/(.*) content.php?r=$1&%{QUERY_STRING}
RewriteRule ^widget/config/([0-9]+) widget.php?r=config/$1&%{QUERY_STRING}
RewriteRule ^$ $1.php?r=$2 [QSA]

My robots.txt

Code:

User-agent: *
Disallow: /tags.php
Disallow: /index.php
Disallow: /arcade.php
Disallow: /editpost.php
Disallow: /inlinemod.php
Disallow: /member.php
Disallow: /memberlist.php
Disallow: /newreply.php
Disallow: /payments.php
Disallow: /printthread.php
Disallow: /private.php
Disallow: /profile.php
Disallow: /faq.php
Disallow: /report.php
Disallow: /search.php
Disallow: /sendmessage.php
Disallow: /showpost.php
Disallow: /usercp.php
Disallow: /usernote.php
Disallow: /chat/
Disallow: /arcade/
Disallow: /ajax.php

Sitemap: http://mysite.com/vbulletin_sitemap_index.xml.gz

Still only 2 pages index, no errors in WMT. Any help would be gratefully recevied.

This was the robots i was running

Code:

User-agent: *
Disallow: /*arcade.php
Disallow: /index.php
Disallow: /*external.php
Disallow: /*attachment.php
Disallow: /*ajax.php
Disallow: /*misc.php
Disallow: /*calendar.php
Disallow: /archive/
Disallow: /*faq.php
Disallow: /*editpost.php
Disallow: /*inlinemod.php
Disallow: /*member.php
Disallow: /*memberlist.php
Disallow: /*newreply.php
Disallow: /*newthread.php
Disallow: /*payments.php
Disallow: /*printthread.php
Disallow: /*private.php
Disallow: /*profile.php
Disallow: /*report.php
Disallow: /*search.php
Disallow: /*sendmessage.php
Disallow: /*usercp.php
Disallow: /*usernote.php
Disallow: /cpstyles/
Disallow: /arcade/
Disallow: /chat/
Disallow: /customavatars/
Disallow: /customcode/
Disallow: /customprofilepics/
Disallow: /images/file_system/
Disallow: /signaturepics/
Disallow: /*tags.php
Disallow: /*.sql$
Disallow: /*.tpl$
Disallow: /blog.php?1-TazDevilLooney&blogcategoryid=1
Disallow: /blog.php?1-TazDevilLooney&tag=how%20to
Disallow: /blog.php?cp=1
Disallow: /blog.php?b=1&goto=next
Disallow: /showthread.php?6-Magic-test&p=0
Disallow: /threadtag.php?do=manage&contenttype=vBBlog_BlogEntry&contentid=1
Disallow: /blogs/1-TazDevilLooney?blogcategoryid=1
Disallow: /blogs/1-TazDevilLooney?m=4&y=2010
Disallow: /blog.php?1-Taz-s-Blog-Log&m=5&y=2010
Disallow: /*group.php?do=viewmembers
Disallow: /content/10-Flash-Game
Disallow: /content.php?13
Disallow: /content.php?18-GunSlinger
Disallow: /list.php?day/1271545201
Disallow: /content.php?10-Flash-Game
Disallow: /content.php?14
Disallow: /content.php?17-Nuns-on-the-run
Disallow: /content.php?15
Disallow: /content.php?16-solitairious

Sitemap: http://mysite.com/vbulletin_sitemap_index.xml.gz

Threaded Mode \| Linear Mode Standard Tools
Forum Jump: View a Printable Version Send this Thread to a Friend Subscribe to this thread